JP4992591B2 - Communication system and communication terminal - Google Patents

Description

  The present invention relates to a communication system and a communication terminal, and more specifically, a communication system that realizes multitalk by connecting three or more communication terminals including sound source localization speakers to each other via a communication line, and the communication system thereof The present invention relates to a communication terminal used for.

  A multi-talk system in which three or more users talk at the same time via a communication terminal has already been realized, and it is expected that it will become more popular in the future as communication technology advances. By the way, when three or more people actually gather and have a conversation, even if a plurality of speakers speak at the same time, the listener can distinguish the speech of each speaker. This phenomenon is called the cocktail party effect, and the human brain uses multiple sound sources from the currently heard sound based on the time difference between the sound reaching the left and right ears, the sense of distance of the sound source, and information obtained from vision. It is said that it has the ability to separate.

  However, unlike the case where three or more people gather and talk in the multi-talk system described above, when a plurality of speakers speak at the same time, simply playing each speaker's voice through the speaker, The time difference between the sound reaching the left and right ears, the sense of distance of the sound source, etc. are made uniform. For this reason, it becomes difficult for the listener to distinguish each speaker's voice, and there arises a problem that the listener cannot understand who is speaking.

In order to solve the above problem, Patent Document 1 describes that a speaker of a communication terminal in a multi-talk system is configured by a stereo speaker and that the sound source of each speaker is localized. To locate a sound source means to arrange each speaker's sound source in a quasi-different position so that the sound of each speaker can be heard from different positions in the three-dimensional sound reproduced from a stereo speaker or the like .
JP 2004-274147 A

  The inventor conducted further research on a multi-talk system in which the sound source of each speaker is localized as described in Patent Document 1. As a result, it was found that even when the sound source of each speaker is localized, when a plurality of speakers speak at the same time, one of the utterances may be blocked by the other, making it difficult to hear.

  In view of the above, the present invention is a communication system that realizes multitalk by connecting three or more communication terminals including a sound source localization speaker to each other via a communication line, and a communication terminal used in the communication system. Thus, it is an object of the present invention to provide a communication system and a communication terminal in which a listener can easily distinguish a speaker's speech even when a plurality of speakers speak simultaneously.

In order to achieve the above object, a communication system according to the first aspect of the present invention provides:
A communication system that realizes multitalk by connecting three or more communication terminals including sound source localization speakers to each other via a communication line,
One or more communication terminals that are transmitting audio signals from the connected communication terminal are identified, and communication state information that associates the identified communication terminals with the volume of the audio signal transmitted by the communication terminals is generated. A communication state detection unit;
A communication terminal that receives the communication status information and receives a currently transmitted voice signal is identified as a receiving terminal, and is currently transmitted based on a sound image synthesis rule set for each identified receiving terminal. A sound image synthesizing device that performs sound image synthesis for each receiving terminal with respect to the audio signal, and transmits the synthesized sound image to the receiving terminal ;
The sound image synthesis rules include rules for sound source localization and volume adjustment,
The sound image synthesis rule stipulates that sound sources of audio signals exceeding a predetermined volume level are arranged isotropically .

The communication system according to the second aspect of the present invention is:
A communication system for realizing videophone multi-talk by connecting three or more communication terminals including a sound source localization speaker and a video display device to each other via a communication line,
Communication state information that identifies one or more communication terminals that are transmitting audio signals and video signals from the connected communication terminal, and associates the identified communication terminals with the volume of the audio signal transmitted by the communication terminal. A communication state detection unit for generating
A communication terminal that receives the communication status information and receives a currently transmitted audio signal and video signal is identified as a receiving terminal, and is currently transmitted based on a video composition rule set for each identified receiving terminal. A video synthesizer that performs video synthesis for each receiving terminal for the received video signal, transmits the synthesized video to the receiving terminal, and generates video synthesis information relating to the synthesized video;
A sound image synthesizing device that performs sound image synthesis for each receiving terminal on the currently transmitted audio signal based on the video synthesis information, and transmits the synthesized sound image to the receiving terminal ;
The video composition rule includes a rule for video layout;
The video synthesis rule defines that video corresponding to an audio signal exceeding a predetermined volume level is isotropically arranged .

  According to the communication system according to the first aspect of the present invention, the sound image synthesizer converts sound signals from other communication terminal devices in a call state into sound images based on the sound image synthesis rules received from the communication terminal devices. Since it is synthesized, it can be synthesized into an optimal sound image that is easy to hear for the listener. Thus, even when a plurality of speakers speak at the same time, the listener can easily distinguish the statements of each speaker.

  According to the communication system according to the second aspect of the present invention, since the video and the sound image correspond to each other, the listener can more easily distinguish the speech of each speaker.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. FIG. 1 is a block diagram of a communication system according to a first embodiment of the present invention. The communication system 10 is a multi-talk system, and includes a communication terminal 11 and a call exchange device 12 connected to the communication terminal 11 via a communication line 13. The call exchange device 12 can realize multitalk by connecting three or more communication terminals 11 simultaneously. The figure shows a state in which four communication terminals 11a to 11d are connected simultaneously.

  FIG. 2 is a block diagram of the communication terminal 11. The communication terminal 11 is configured as, for example, a mobile phone, and includes an input device 21 that receives user input operations, a display device 22 that includes a liquid crystal display panel, a storage device 23, a communication device 24, and a receiver device 25 that includes a speaker. , A transmission device 26 composed of a microphone, and a control device 27.

  The receiver 25 is configured as a sound source localization speaker. The sound source localization speaker is generally called a stereo speaker or a surround speaker, and is a speaker or a speaker set that is reproduced so that a plurality of sound sources can be heard from different directions and distances in a pseudo manner. The communication device 24 transmits and receives audio signals to and from the call exchange device 12. The control device 27 includes a temporary storage device and a central processing unit, operates according to a predetermined control program, and controls the operations of the units 21 to 26 in the communication terminal 11.

  FIG. 3 is a block diagram of the call exchange device 12. The call exchange device 12 includes a communication device 31 that transmits and receives audio signals to and from the plurality of communication terminals 11, a storage device 32, and a control device 33. The control device 33 includes a temporary storage device and a central processing unit, operates according to a predetermined control program, and controls the operations of the units 31 and 32 in the call exchange device 12.

  The control device 33 includes a call state detection unit 34 that detects a current call state, and a sound image synthesis processing unit (sound image synthesis device) 35 that synthesizes a sound signal received from each communication terminal 11 with a sound image. Synthesizing with a sound image means that in the three-dimensional sound reproduced from the sound source localization speaker, each speaker's sound source is placed in a pseudo different position so that each speaker's sound can be heard from different positions (sound source localization). ), Or adjusting the volume of each speaker (volume adjustment), or both.

  FIG. 4 is a flowchart showing a procedure of a call method using the communication system 10 of FIG. The sound image synthesis rules are stored in advance in the storage device 23 of the communication terminal 11 (step S11). The synthesis of sound images is to synthesize the sound signals of a plurality of speakers into one sound image, for example, the number of people currently in a call state, the number of current utterances, the loudness of utterances, This is based on information related to the person in a call state. The sound image synthesis rule defines a rule for the sound image synthesis.

  The sound image synthesis rule can be preset by the user using the input device 21 and can be changed even during a call. Alternatively, some sound image synthesis rule candidates may be stored in the storage device 23 in advance, and the user can select from these candidates via the input device 21.

  When three or more communication terminals 11 including the own communication terminal 11 are simultaneously connected and multitalk is realized, the control device 27 of the communication terminal 11 reads the sound image synthesis rule from the storage device 23, and the communication device 24. Is transmitted to the call exchange device 12 (step S12). The control device 33 of the call exchange device 12 stores the received sound image synthesis rule in the storage device 32 and activates the call state detection unit 34 and the sound image synthesis processing unit 35 (step S13).

  The call state detecting unit 34 detects the current call state and associates the identification information of each communication terminal 11 connected to the communication terminal 11 that has received the sound image synthesis rule with the volume information of the audio signal to communicate the communication state. Information is generated and output to the sound image synthesis processing unit 35 (step S14). The sound image synthesis processing unit 35 synthesizes the audio signal received from each communication terminal 11 into a sound image based on the communication state information and the sound image synthesis rule obtained from the call state detection unit 34 (step S15). In synthesizing a sound image, either or both of sound source localization and volume adjustment of an audio signal received from each communication terminal 11 is performed.

  The sound image synthesis processing unit 35 transmits the synthesized sound image to the communication terminal 11 that has received the sound image synthesis rules via the communication device 31 (step S16). The control device 27 of the communication terminal 11 receives the sound image from the call exchange device 12, and reproduces the received sound image from the receiver device 25 (step S17). By reproducing the sound image with sound source localization and volume adjustment, it is heard by the listener that each speaker is speaking at a different position, and the speaker's speech can be distinguished.

  FIGS. 5A and 5B show examples of sound image synthesis rules. When synthesizing sound images, in principle, a plurality of sound sources are arranged as isotropic as possible. “Isotropically arranged” means that a plurality of sound sources are arranged in symmetrical directions around a listener in a one-dimensional, two-dimensional, or three-dimensional space.

  For example, when two sound sources are arranged, as shown in FIG. 5A, the two sound sources are arranged on a straight line intersecting the listener so as to be symmetric with respect to the listener. When three sound sources are arranged, as shown in FIG. 5B, the three sound sources are arranged at the apex of an equilateral triangle centered on the listener. When four sound sources are arranged, the four sound sources are arranged at the vertices of a square or a regular tetrahedron centered on the listener. Thus, by arranging a plurality of sound sources isotropically, the listener can easily distinguish the speech of each speaker.

  As another example of the sound image synthesis rule, the distance between the listener and the sound source may be set after the direction of the sound source is determined as shown in FIGS. For example, based on the volume of each speaker, a loud sound source may be placed near the listener, and a low sound source may be placed far from the listener.

  As another example of the sound image synthesis rule, a speech signal whose volume exceeds a certain level may be extracted as a speech, and only the speech signal extracted as the speech may be synthesized with a sound source. Compared with the case where the arrangement of each speaker is fixed, the utterances can be arranged apart from each other, so that the utterance of each speaker can be more easily distinguished by the listener. In this case, you may adjust so that the volume of each utterance may become uniform.

  As another example of the sound image synthesis rule, when the volume of an audio signal exceeds a level B that is further higher than a level A that is regarded as an utterance, the utterance is regarded as an utterance with a high volume. Further, after the messages whose volume exceeds level B are preferentially arranged isotropically, the messages whose volume does not exceed level B are as isotropic as possible between the positions where the messages whose volume exceeds level B are arranged. You may arrange in.

  As another example of the sound image synthesis rule, the other party is specified by the telephone book function of the communication terminal 11 and related information of the other party is extracted from the telephone book data. Moreover, based on the extracted related information of the other party, the other party is grouped or ranked. Further, sound source localization and volume adjustment of audio signals by each call partner are performed for each group or based on the order.

  As another example of the sound image synthesis rule, the other party is specified by the telephone directory function of the communication terminal 11, and information such as the number of past calls with the other party and the duration of the call is recorded from the call record. Extract information such as the number of past emails sent and received and the size of emails sent and received. Further, based on the extracted information, the call partners are grouped or ranked. Further, sound source localization and volume adjustment of audio signals by each call partner are performed for each group or based on the order.

  When the sound image synthesis rule stipulates that the telephone book function or the like of the communication terminal 11 is used as described above, the control device 27 of the communication terminal 11 prior to the transmission of the sound image synthesis rule in step S12, Use the phone book function, etc., and make detailed settings for sound image synthesis rules. The sound image synthesis rule may be configured by combining the above-described examples. In addition to the examples described above, various modes can be adopted according to the communication system 10 and the purpose of the user.

  When the user changes the sound image synthesis rule, the changed new sound image synthesis rule is stored in the storage device 32 of the call exchange device 12 according to steps S12 and S13. In step S14, the sound image synthesis processing unit 35 of the call exchange device 12 is stored. Synthesizes a sound image based on a new sound image synthesis rule.

  One example of a method for changing the sound image synthesis rule during a call is to select a sound image synthesis rule that is different from the currently selected sound image synthesis rule from a plurality of sound image synthesis rules stored in the storage device 23 of the communication terminal 11. I can do it. In addition, the positions of the sound sources of the speakers can be interchanged. Further, the volume level of each speaker can be switched between each other. Furthermore, the relationship between the positions of the sound sources of the speakers and the relationship between the loudness levels of the speakers can be switched to each other.

  In this embodiment, since the listener can change the sound image synthesis rule during a call, the sound image synthesis rule can be applied even when the listener feels it is difficult to distinguish each speaker's utterance with the currently applied sound image synthesis rule. By changing the composition rule, it can be made easier to distinguish. In addition, even when the currently applied sound image synthesis rules do not match the listener's intention, the listener can be changed to a desired one, thereby preventing the listener from feeling inconvenience.

  For example, if two speakers are speaking at the same time during multi-talk, and the sound sources of these two speakers are located close to each other, the listener will speak each speaker. Difficult to distinguish. In this case, if the sound sources of the two speakers are arranged at positions far from each other, it is possible to easily distinguish the speech of each speaker.

  In addition, when a plurality of speakers are speaking at the same time, the listener wants to pay attention to the speech of a specific speaker, but may be difficult to hear due to being blocked by the speech of another speaker. In this case, if the arrangement of the sound sources of each speaker is changed or the volume of each speaker is adjusted so that the speech of this specific speaker is easy to hear, the above-mentioned speech can be easily heard. Furthermore, by adjusting the pseudo distance between each speaker's sound source and the listener in accordance with the volume of each speaker, each speaker's utterances can be more easily distinguished and enjoy a realistic call. I can do it.

  According to the communication system 10 of the present embodiment, the sound image synthesis processing unit 35 synthesizes audio signals from the other communication terminals 11 in a call state based on the sound image synthesis rules received from the communication terminals 11 into sound images. Therefore, it can be synthesized into an optimal sound image that is easy for the listener to distinguish. As a result, even when a plurality of speakers speak at the same time, it is easy for the listener to distinguish the speech of each speaker.

  In the communication system 10 of the first embodiment, it is not necessary that both the call state detection unit 34 and the sound image synthesis processing unit 35 are provided in the call exchange device 12, for example, either one or both are communication terminals. 11 control devices 27.

  The communication system according to the second embodiment of the present invention is configured as a videophone multitalk system. The communication terminal 11 has a photographing device (not shown) in addition to the various functions shown in FIG. The communication device 24 of the communication terminal 11 performs shooting during a call and transmits the captured video information to the call exchange device 12. In addition, video information is received from the call exchange device 12 during a call and displayed on the display device 22.

  FIG. 6 is a block diagram of the control device 33 of the call exchange device 12 in this embodiment. In addition to the call state detection unit 34 and the sound image synthesis processing unit 35, the control device 33 of the call exchange device 12 includes a video synthesis processing unit (video synthesis device) 36 that synthesizes video received from each communication terminal 11 in the call state. Have

  FIG. 7 is a flowchart showing a procedure of a call method using the communication system of the present embodiment. The video composition rule is stored in advance in the storage device 23 of the communication terminal 11 (step S21). Video synthesis is to combine a plurality of speakers' videos into one video, and the video synthesis rule defines rules for video synthesis. Similarly to the above-described sound image synthesis rule, the video synthesis rule can also be set in advance by the listener using the input device 21 and can be changed even during a call. Alternatively, some candidates for the video composition rule may be stored in advance in the storage device 23 so that a listener can select from these candidates via the input device 21.

  The control device 27 of the communication terminal 11 reads the video composition rule from the storage device 23 when three or more communication terminals 11 including the own communication terminal 11 are connected at the same time and videophone multitalk is realized. The message is transmitted to the call exchange device 12 via the device 24 (step S22). The control device 33 of the call exchange device 12 stores the received video composition rule in the storage device 32, and activates the call state detection unit 34, the sound image synthesis processing unit 35, and the video synthesis processing unit 36 (step S23). .

  The call state detection unit 34 detects the current call state and associates the identification information of each communication terminal 11 connected to the communication terminal 11 that has received the video composition rule with the information on the volume of the audio signal. Information is generated and output to the video composition processing unit 36 (step S24).

  The video composition processing unit 36 determines part or all of the arrangement, size, movement, and display state of the video received from each communication terminal 11 based on the communication state information and the video composition rule obtained from the call state detection unit 34. Based on this determination, the video received from each communication terminal 11 is synthesized (step S25). The video composition processing unit 36 transmits the information of the synthesized video to the communication terminal 11 that has received the video composition rule, and the identification information of each communication terminal 11 and the arrangement and size of the video received from each communication terminal 11. Video synthesis information is generated by associating the movement and display state information, and is output to the sound image synthesis processing unit 35 (step S26).

  The sound image synthesis processing unit 35 synthesizes the sound image received from each communication terminal 11 based on the video synthesis information (step S27). The sound image synthesis processing unit 35 transmits information of the synthesized sound image to the communication terminal 11 that has received the video synthesis rule (step S28). The communication terminal 11 receives the video information from the video synthesis processing unit 36 and displays the video on the display device 22 (step S29), and also receives the sound image information from the sound image synthesis processing unit 35 and reproduces the sound image on the receiver 25 (step S30). ).

  As an example of the video composition rule, the size and arrangement of the corresponding video may be set according to the volume of the audio signal. In this case, the video of a specific speaker may be displayed so as to be overlaid on the video of another speaker. As another example of the video synthesis rule, a voice signal whose volume exceeds a certain level may be extracted as a utterance, and the video of the speaker who is speaking may be displayed relatively large. Alternatively, only the video of the speaker who is speaking may be arranged on the screen. In these cases, the video of the speaker who is speaking may be arranged isotropically on the screen.

  As another example of the video composition rule, the video of each speaker is placed at the edge of the screen or is not displayed, and it is regarded as a speech when the volume of the audio signal exceeds a certain level. The video may be moved and displayed from the edge of the screen toward the center. In this case, the video of the speaker who is speaking may be arranged isotropically near the center of the screen.

  As another example of the video composition rule, when the volume of the audio signal exceeds a level B that is further higher than the level A that is regarded as a speech, the speech is regarded as a speech with a high volume. Furthermore, after the video corresponding to the speech whose volume exceeds level B is preferentially arranged, the video corresponding to the speech whose volume does not exceed level B is displayed, and the video corresponding to the speech whose volume exceeds level B. It may be arranged as isotropic as possible between the arrangement locations.

  As another example of the video composition rule, the other party is specified by the telephone book function of the communication terminal 11 and related information of the other party is extracted from the telephone book data. Moreover, based on the extracted related information of the other party, the other party is grouped or ranked. Further, the arrangement, size, movement, and display state of each caller's video are set for each group or based on the order.

  As another example of the video composition rule, the other party is specified by the telephone directory function of the communication terminal 11, and information such as the number of past calls with the other party from the call record and the call duration are obtained from the mail transmission / reception record. Extract information such as the number of past emails sent and received and the size of emails sent and received. Further, based on the extracted information, the call partners are grouped or ranked. Further, the arrangement, size, movement, and display state of each caller's video are set for each group or based on the order.

  As a result, even when a plurality of speakers speak at the same time, the visual effect is combined, and the listener can more easily distinguish each speaker's speech. When the video composition rule specifies that the telephone book function or the like of the communication terminal 11 is used as described above, the control device 27 of the communication terminal 11 prior to the transmission of the video composition rule in step S22. Then, use the phone book function, etc., and make detailed settings for video composition rules. The video composition rule may be configured by combining the above-described examples. In addition to the example described above, various modes can be adopted for the video composition rule depending on the communication system and the purpose of the user.

  When synthesizing the sound image in step S27, for example, the arrangement and direction of the sound sources of the speakers can be set so as to correspond to the arrangement of the images of the speakers. In this case, since the video and the sound image match, when a plurality of speakers speak at the same time, the listener can more easily distinguish the speech of each speaker.

  According to the communication system of the present embodiment, video and multi-talk can be realized so that a video and a sound image can be associated with each other, so that even when a plurality of speakers speak at the same time, visual communication is possible. Combined with the effects, the listener can easily distinguish each speaker's speech.

  In the communication system of the second embodiment, it is not necessary that the call state detection unit 34, the sound image synthesis processing unit 35, and the video synthesis processing unit 36 are all provided in the call exchange device 12. For example, any of them may be arranged in the control device 27 of the communication terminal 11.

  Although the present invention has been described based on the preferred embodiment, the communication system and communication terminal of the present invention are not limited to the configuration of the above embodiment, and various modifications can be made from the configuration of the above embodiment. Further, modifications and changes are also included in the scope of the present invention.

1 is a block diagram of a communication system according to a first embodiment of the present invention. It is a block diagram of the communication terminal of FIG. FIG. 2 is a block diagram of the call exchange device of FIG. 1. It is a flowchart which shows the operation | movement procedure of the communication system of FIG. FIGS. 5A and 5B are diagrams schematically illustrating an example of a sound image synthesis rule. It is a block diagram of the control apparatus of a speech exchange apparatus about the communication system which concerns on 2nd Example of this invention. It is a flowchart which shows the operation | movement procedure of the communication system which concerns on 2nd Example of this invention.

Explanation of symbols

10: communication system 11, 11a to 11d: communication terminal 12: call exchange device 13: communication line 21: input device 22: display device 23: storage device 24: communication device 25: receiver device 26: transmitter device 27: control device 31: Communication device 32: Storage device 33: Control device 34: Call state detection unit 35: Sound image synthesis processing unit 36: Video synthesis processing unit

Claims (23)

A communication system that realizes multitalk by connecting three or more communication terminals including sound source localization speakers to each other via a communication line,
One or more communication terminals that are transmitting audio signals from the connected communication terminal are identified, and communication state information that associates the identified communication terminals with the volume of the audio signal transmitted by the communication terminals is generated. A communication state detection unit;
A communication terminal that receives the communication status information and receives a currently transmitted voice signal is identified as a receiving terminal, and is currently transmitted based on a sound image synthesis rule set for each identified receiving terminal. A sound image synthesizing device that performs sound image synthesis for each receiving terminal with respect to the audio signal, and transmits the synthesized sound image to the receiving terminal ;
The sound image synthesis rules include rules for sound source localization and volume adjustment,
The communication system according to claim 1, wherein the sound image synthesis rule defines that sound sources of audio signals exceeding a predetermined volume level are arranged isotropically . The communication system according to claim 1 , wherein the sound image synthesis rule defines that the volume of an audio signal exceeding a predetermined volume level is uniform. The communication system according to claim 1 or 2 , wherein the sound image synthesis rule defines that a sound source distance of an audio signal corresponds to a sound volume. The communication system according to claim 1 , wherein the sound image synthesis rule defines that the audio signal is sorted into a plurality of stages according to the sound volume, and a sound source of the sound signal is arranged at each of the sorted stages. A communication system for realizing videophone multi-talk by connecting three or more communication terminals including a sound source localization speaker and a video display device to each other via a communication line,
Communication state information that identifies one or more communication terminals that are transmitting audio signals and video signals from the connected communication terminal, and associates the identified communication terminals with the volume of the audio signal transmitted by the communication terminal. A communication state detection unit for generating
A communication terminal that receives the communication status information and receives a currently transmitted audio signal and video signal is identified as a receiving terminal, and is currently transmitted based on a video composition rule set for each identified receiving terminal. A video synthesizer that performs video synthesis for each receiving terminal for the received video signal, transmits the synthesized video to the receiving terminal, and generates video synthesis information relating to the synthesized video;
A sound image synthesizing device that performs sound image synthesis for each receiving terminal on the currently transmitted audio signal based on the video synthesis information, and transmits the synthesized sound image to the receiving terminal ;
The video composition rule includes a rule for video layout;
The communication system characterized in that the video composition rule defines that video corresponding to an audio signal exceeding a predetermined volume level is isotropically arranged . The communication system according to claim 5 , wherein the video composition rule defines that only video corresponding to an audio signal exceeding a predetermined volume level is displayed. The communication system according to claim 5 or 6 , wherein the video composition rule defines that a size of a corresponding video is set according to a volume of an audio signal. The video compositing rules, and sorting the plurality of stages depending on the volume of the audio signal exceeds a predetermined volume level, to define the effect of placing an image corresponding to the audio signal for each stage of the sorting claim 5 The communication system according to 1. A communication terminal used in the communication system according to any one of claims 1 to 4 ,
A storage device for storing sound image synthesis rules;
A control device that reads and transmits a sound image synthesis rule from the storage device;
A communication terminal comprising: The communication terminal according to claim 9 , wherein the sound image synthesis rules stored in the storage device can be set or changed via an input device. The storage device stores a plurality of sound image synthesis rule candidates,
The communication terminal according to claim 9 , wherein a desired sound image synthesis rule is selectable from a plurality of sound image synthesis rule candidates stored in the storage device via an input device. The controller is
A function for identifying a user of a communication terminal connected via a communication line using a telephone directory function, and extracting information of the specified user;
A function of setting the sound image synthesis rule based on the information of the identified user;
The communication terminal according to claim 9 , comprising: The information of the specified user is the number of past calls with the specified user, the call time, the number of mail transmission / reception, or the size of the mail transmitted / received,
13. The communication terminal according to claim 12 , wherein the control device groups or ranks the plurality of identified users when setting the sound image synthesis rule. A communication terminal used in the communication system according to any one of claims 5 to 8 ,
A storage device for storing video composition rules;
A control device that reads and transmits a video composition rule from the storage device;
A communication terminal comprising: Stored video combining rule in the storage device, via the input device is configured to be set or changed, the communication terminal according to claim 1 4. The storage device stores a plurality of video synthesis rule candidates,
From the candidates of the storage device a plurality of video composition rules stored in, via the input device, a desired image synthesis rules are configured to be selected, the communication terminal according to claim 1 4. The controller is
A function for identifying a user of a communication terminal connected via a communication line using a telephone directory function, and extracting information of the specified user;
A function of setting the video composition rule based on the specified user information;
The a communication terminal according to claim 1 4. The information of the specified user is the number of past calls with the specified user, the call time, the number of mail transmission / reception, or the size of the mail transmitted / received,
The communication terminal according to claim 17 , wherein the control device groups or ranks the plurality of identified users when setting the video composition rule. A communication terminal used in the communication system according to any one of claims 1 to 4 ,
A sound image synthesizer is provided that receives a sound signal and communication state information transmitted from a connected communication terminal, and performs sound image synthesis on a currently transmitted sound signal based on a sound image synthesis rule set in the terminal. A communication terminal characterized by that. A communication terminal used in the communication system according to any one of claims 5 to 8 ,
A video synthesizing device that receives a video signal and communication state information transmitted from a connected communication terminal and performs video synthesis on a currently transmitted video signal based on a video synthesis rule set in the terminal is provided. A communication terminal characterized by that. A communication terminal used in the communication system according to any one of claims 5 to 8 ,
It comprises a sound image synthesizer that receives a video signal and video synthesis information transmitted by a connected communication terminal and performs sound image synthesis on a currently transmitted audio signal based on the received video synthesis information. Communication terminal. A communication terminal used in a communication system,
The communication system is:
Multi-talk is realized by connecting three or more communication terminals equipped with sound source localization speakers to each other via a communication line.
One or more communication terminals that are transmitting audio signals from the connected communication terminal are identified, and communication state information that associates the identified communication terminals with the volume of the audio signal transmitted by the communication terminals is generated. A communication state detection unit;
A communication terminal that receives the communication status information and receives a currently transmitted voice signal is identified as a receiving terminal, and is currently transmitted based on a sound image synthesis rule set for each identified receiving terminal. A sound image synthesizing device that performs sound image synthesis for each receiving terminal with respect to the audio signal, and transmits the synthesized sound image to the receiving terminal;
The communication terminal is
A storage device for storing sound image synthesis rules;
A control device that reads and transmits a sound image synthesis rule from the storage device, and
The controller is
A function for identifying a user of a communication terminal connected via a communication line using a telephone directory function, and extracting information of the specified user;
A function of setting the sound image synthesis rule based on the specified user information,
The information of the specified user is the number of past calls with the specified user, the call time, the number of mail transmission / reception, or the size of the mail transmitted / received,
Communication terminal and the control device, which upon setting of the sound image combining rule, grouping a plurality of users who have the particular or ranks, and wherein the. A communication terminal used in a communication system,
The communication system is:
Three or more communication terminals including a sound source localization speaker and a video display device are connected to each other via a communication line to realize a videophone multitalk.
Communication state information that identifies one or more communication terminals that are transmitting audio signals and video signals from the connected communication terminal, and associates the identified communication terminals with the volume of the audio signal transmitted by the communication terminal. A communication state detection unit for generating
A communication terminal that receives the communication status information and receives a currently transmitted audio signal and video signal is identified as a receiving terminal, and is currently transmitted based on a video composition rule set for each identified receiving terminal. A video synthesizer that performs video synthesis for each receiving terminal for the received video signal, transmits the synthesized video to the receiving terminal, and generates video synthesis information relating to the synthesized video;
A sound image synthesizing device that performs sound image synthesis for each receiving terminal on the currently transmitted audio signal based on the video synthesis information, and transmits the synthesized sound image to the receiving terminal;
The communication terminal is
A storage device for storing video composition rules;
A control device that reads and transmits a video composition rule from the storage device, and
The controller is
A function for identifying a user of a communication terminal connected via a communication line using a telephone directory function, and extracting information of the specified user;
A function of setting the video composition rule based on the information of the identified user,
The information of the specified user is the number of past calls with the specified user, the call time, the number of mail transmission / reception, or the size of the mail transmitted / received,
Communication terminal and the control device, in which when setting of the video combining rule, grouping a plurality of users who have the particular or ranks, and wherein the.

Images