JP2011066730A - Communication terminal, communication method, and communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a communication terminal, a communication method, and a communication system, wherein a network band usable among communication terminals of users who have conversation is efficiently secured without increasing the network band itself usable among a plurality of communication terminals which participate in a conference when the users have the conversation. <P>SOLUTION: The communication terminal detects that the conversation starts when the conversation starts between the users of two communication terminals. A conversation terminal which is a communication terminal of a user who starts the conversation specifies a partner terminal which is a terminal of a conversation partner (S2). When the conversation starts (S4: YES), the conversation terminal selects a non-conversation terminal which does not have the conversation (S6). The conversation terminal transmits communication data only to the selected non-conversation terminal and the partner terminal (S8, S9). The non-conversation terminal selected from the conversation terminal transmits the communication data to other non-conversation terminals (S15). <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a communication terminal, a communication method, and a communication system for performing a conference by transmitting / receiving data including at least one of audio data and image data between a plurality of communication terminals.

  2. Description of the Related Art Conventionally, a technique for transmitting and receiving audio data and image data between a plurality of communication terminals is known so that a conference can be executed even when all of a plurality of users are not at the same base. When users are having a conversation, in order to secure a network bandwidth that can be used between communication terminals of the users who are having a conversation, for example, the video communication device described in Patent Document 1 has a conversation between users. Different networks are used depending on whether or not

JP-A-7-222129

  In the technique described in Patent Literature 1, the network bandwidth itself that can be used between a plurality of communication terminals participating in a conference is increased. In this case, there is a problem that the network bandwidth that can be used by other devices not participating in the conference is compressed.

  The present invention enables communication between communication terminals of users who are having a conversation without increasing the network bandwidth that can be used between a plurality of communication terminals participating in the conference when the users are having a conversation. It is an object of the present invention to provide a communication terminal, a communication method, and a communication system that can efficiently secure a usable network band.

  The communication terminal according to the first aspect of the present invention is a communication terminal that connects to three or more other communication terminals via a network, and transmits and receives communication data including at least one of audio data and image data. The first start detecting means for detecting that the conversation has started between the user of the own terminal and the user of one communication terminal among the other communication terminals, and the conversation is started by the first start detecting means. A first identification unit that identifies the partner terminal that is the one communication terminal on the partner side of the conversation that is started, and the partner terminal is identified by the first identification unit Communication data from selection means for selecting at least one non-conversation terminal excluding the counterpart terminal that is having a conversation among the other communication terminals, and information including at least one of voice data and image data Selected by the generating means, the first transmitting means for transmitting the first data, which is the communication data generated by the generating means, to the counterpart terminal specified by the first specifying means, and the selecting means Second communication means for transmitting to the non-conversation terminal second data whose communication source information is the same as the first data, and two communication terminals among the other communication terminals The second start detecting means for detecting that the conversation has started between the users, and the second start detecting means, when the second start detecting means detects that the conversation has been started, A second specifying means for specifying one communication terminal, and when receiving second data from the other communication terminal, a communication terminal other than the two communication terminals specified by the second specifying means, And a transfer means for transferring said second data signal.

  According to the communication terminal according to the first aspect, among the plurality of communication terminals, between the two communication terminals (hereinafter referred to as “conversation terminals”) used by the user having a conversation, the first data is mutually exchanged. Sent and received. The conversation terminal transmits the second data to at least one of the plurality of non-conversation terminals. The non-conversation terminal that has received the second data transfers the second data to another non-conversation terminal. Therefore, during a conversation, the conversation terminal can execute a conference without transmitting communication data to all of the other communication terminals. Therefore, the network bandwidth that can be used by the conversation terminal can be secured by efficiently using the network bandwidth without increasing the network bandwidth itself that can be used between the plurality of communication terminals. As a result, since communication data is stably transmitted and received between the two conversation terminals, the conference can proceed smoothly. Communication data having a large amount of data can be transmitted and received between conversation terminals.

  The first transmission unit may transmit first data having a larger data amount than the second data transmitted by the second transmission unit to the counterpart terminal. In this case, when the communication terminal operates as a conversation terminal, the communication terminal transmits the first data having a data amount larger than the second data to the partner terminal using the bandwidth secured with the partner terminal on the partner side of the conversation. can do. Therefore, the communication terminal can provide at least one of audio and video to a user who is in conversation with high quality. Further, the communication terminal transmits second data having a data amount smaller than that of the first data to the non-conversation terminal. Therefore, a band that can be used with the counterpart terminal can be more reliably secured, and the first data can be smoothly transmitted to the counterpart terminal.

  The communication terminal may further include storage control means for sequentially storing the counterpart terminal specified by the first specifying means as a history in the first storage means. The selection means may include first selection means for preferentially selecting a non-conversation terminal having a new order of history stored in the first storage means among the plurality of non-conversation terminals. The delay time from when the second data is transmitted from the conversation terminal until it is received by the non-conversation terminal is shorter when the second data is transmitted directly from the conversation terminal to the non-conversation terminal than when it is transferred. Therefore, in this case, communication data can be transmitted with minimal time delay to a conversation partner that is generally highly relevant to the current conversation and has a new history storage order.

  The selection means is a second selection means for preferentially selecting a non-conversation terminal having a short transmission time of communication data with the own terminal among the plurality of non-conversation terminals, and a plurality of the non-conversation terminals. A third selection means for preferentially selecting a non-conversation terminal having a high network bandwidth with the terminal, and a non-conversation terminal having a high data processing capacity among the plurality of non-conversation terminals. It is desirable to include at least one of the fourth selection means. In this case, when operating as a conversation terminal, the communication terminal transmits the second data to the non-conversation terminal that has not been selected by the selection means in a state where transmission delay is low or a problem during transmission is unlikely to occur. can do.

  The communication terminal further includes first acquisition means for acquiring the priority from second storage means for storing the priority of each communication terminal in a plurality of communication terminals connected via the network. Good. The selection unit may include a fifth selection unit that preferentially selects a non-conversation terminal having a high priority acquired by the first acquisition unit among the plurality of non-conversation terminals. In this case, by setting the priority of each communication terminal, the user can transmit communication data to the user of the communication terminal having a high priority while minimizing the time delay.

  The communication terminal obtains the specific information from third storage means for storing specific information for specifying a user who uses each of the communication terminals in association with the communication terminal used by the user. Means, a third acquisition means for acquiring the speech voice of the user who uses each of the plurality of communication terminals, and a voice recognition means for recognizing the utterance content from the speech voice acquired by the third acquisition means. May be. The first specifying unit may include a voice specifying unit that specifies the partner terminal using the specific information when the specific information is included in the utterance content recognized by the voice recognition unit. In this case, the communication terminal can easily specify the partner terminal that is the communication terminal of the conversation partner based on the user's speech.

  The communication terminal includes a display control unit that displays a user of the other communication terminal on a display unit that displays an image based on communication data transmitted from the other communication terminal, and an imaging unit that captures the user. You may further provide the 4th acquisition means which acquires the said user's captured image, and the gaze detection means which detects the said user's gaze direction from the captured image acquired by the said 4th acquisition means. The first identification unit identifies a communication terminal used by the user in the line-of-sight direction as the partner terminal when the user displayed on the display unit is in the line-of-sight direction detected by the line-of-sight detection unit. Specific means may be provided. In this case, the communication terminal can easily identify the partner terminal that is the communication terminal of the conversation partner based on the user's line of sight.

  The communication terminal detects whether the conversation between the users of the two communication terminals has ended after the first start detection means or the second start detection means detects that the conversation has started. A third means for transmitting third data, which is communication data generated by the generating means, to all of the other communication terminals when the end of the conversation is detected by the detecting means and the end detecting means; It is desirable to further comprise a transmission means. When the communication between the users of the two communication terminals among the plurality of communication terminals including the self terminal is completed, the communication terminals configured as described above can directly transmit and receive communication data between the terminals. Therefore, when a conversation between the two terminals is not performed, a time delay from when the communication data is transmitted until it is received can be minimized.

  A communication method according to a second aspect of the present invention is performed by a communication system including four or more communication terminals that are connected to each other via a network and transmit / receive communication data including at least one of audio data and image data. A first start detection step of detecting that a conversation has started between a user of the own terminal and a user of one communication terminal among the other communication terminals connected to the own terminal; A first specifying step of specifying a partner terminal that is the one communication terminal on the partner side of the started conversation when the first start detecting step detects that the conversation is started; and A selection step of selecting at least one non-conversation terminal excluding the counterpart terminal having a conversation among the other communication terminals when the counterpart terminal is identified by the identification step; A generation step of generating communication data from information including at least one of voice data and image data, and first data which is communication data generated by the generation step are added to the counterpart terminal specified by the first specification step. A first transmission step for transmitting, and a second data for transmitting the second data, which is communication data common to the first data, to the non-conversation terminal selected in the selection step. A transmission step, a second start detection step for detecting that a conversation is started between users of two communication terminals in the other communication terminals, and a detection that the conversation is started by the second start detection step A second specifying step of specifying the two communication terminals having a conversation that has been started, and the other communication When receiving the second data from the end, in the second specific communication terminal other than the two communication terminals identified by the step, and a transfer step of transferring the second data received.

  According to the communication method according to the second aspect, among the plurality of communication terminals, the first data is transmitted and received between two conversation terminals used by the user having a conversation. The conversation terminal transmits the second data to at least one of the plurality of non-conversation terminals. The non-conversation terminal that has received the second data transfers the second data to another non-conversation terminal. Therefore, during a conversation, the conversation terminal can execute a conference without transmitting communication data to all of the other communication terminals. Therefore, the network bandwidth that can be used by the conversation terminal can be secured by efficiently using the network bandwidth without increasing the network bandwidth itself that can be used between the plurality of communication terminals. As a result, since communication data is stably transmitted and received between the two conversation terminals, the conference can proceed smoothly. Communication data having a large amount of data can be transmitted and received between conversation terminals.

  A communication system according to a third aspect of the present invention includes four or more communication terminals that transmit and receive communication data including at least one of audio data and image data, and the plurality of communication terminals are connected to each other via a network. A first start detecting means for detecting that a conversation is started between a user of the own terminal and a user of one communication terminal among the other communication terminals; A first specifying means for specifying a partner terminal that is the one communication terminal on the partner side of the started conversation when the first start detecting means detects that the conversation is started; and Selecting means for selecting at least one non-conversation terminal excluding the counterpart terminal having a conversation among the other communication terminals when the counterpart terminal is specified by the specifying means; A generating unit that generates communication data from information including at least one of the image data; and a first data that is the communication data generated by the generating unit is transmitted to the counterpart terminal specified by the first specifying unit. And a second transmission means for transmitting, to the non-conversation terminal selected by the selection means, second data that is communication data in which the information generated by the generation means is common to the first data. When it is detected that the conversation is started by the second start detecting means for detecting that the conversation is started between the users of the two communication terminals in the other communication terminals, and the second start detecting means. The second specifying means for specifying the two communication terminals that have started the conversation and the second specifying means when the second data is received from the other communication terminals. The communication terminal other than the two communication terminals identified I, and a transfer means for transferring said received second data.

  According to the communication system according to the third aspect, among the plurality of communication terminals, the first data is transmitted and received between the two conversation terminals used by the user having a conversation. The conversation terminal transmits the second data to at least one of the plurality of non-conversation terminals. The non-conversation terminal that has received the second data transfers the second data to another non-conversation terminal. Therefore, during a conversation, the conversation terminal can execute a conference without transmitting communication data to all of the other communication terminals. Therefore, the network bandwidth that can be used by the conversation terminal can be secured by efficiently using the network bandwidth without increasing the network bandwidth itself that can be used between the plurality of communication terminals. As a result, since communication data is stably transmitted and received between the two conversation terminals, the conference can proceed smoothly. Communication data having a large amount of data can be transmitted and received between conversation terminals.

1 is a diagram illustrating a system configuration of a video conference system 3. FIG. 2 is a block diagram showing an electrical configuration of the video conference terminal 1. FIG. It is a figure which shows an example of the image displayed on the display apparatus 34 during a video conference. 4 is a schematic diagram showing information stored in a communication partner information storage area 132 of the HDD 13. FIG. 3 is a schematic diagram showing information stored in a conversation history storage area 133 of the HDD 13. FIG. It is a flowchart of the main process which the video conference terminal 1 and PC2 perform. It is a flowchart of the meeting start process performed during a main process. It is a flowchart of the 1st conversation start detection process which the video conference terminal 1 and PC2 perform. It is a flowchart of the 2nd conversation start detection process which the video conference terminal 1 and PC2 perform. It is a flowchart of the relay terminal selection process performed during the main process. It is a flowchart of the relay terminal acceptance process which the video conference terminal 1 and PC2 perform. It is a figure which shows an example of the transmission / reception aspect of the normal data in the video conference system 3. FIG. It is a figure which shows an example of the transmission / reception aspect of data in case the user of the base A and the user of the base E are having conversation. It is a figure which shows the transmission / reception aspect of the communication terminal of the base A in the case of FIG.

  Hereinafter, a video conference terminal 1 and a personal computer (hereinafter referred to as “PC”) 2 which are embodiments of a communication terminal according to the present invention will be described with reference to the drawings. The drawings to be referred to are used for explaining technical features that can be adopted by the present invention. The configuration of the apparatus, the flowcharts of various processes, and the like described in the drawings are not intended to be limited to these, but are merely illustrative examples.

  With reference to FIG. 1, a system configuration of a video conference system 3 including a video conference terminal 1 and a PC 2 will be described. The video conference system 3 includes a plurality of communication terminals connected via a network 8. In FIG. 1, as a communication terminal constituting the video conference system 3, a video conference terminal 1 arranged at the base A, a PC 2 arranged at the base B, and a video conference terminal 1 arranged at the base C are illustrated. . In the video conference system 3, each communication terminal inputs / outputs information to / from each other, thereby allowing users at a plurality of bases to share video and audio. As a result, even when all the users are not at the same base, the users can smoothly execute the conference.

  Each communication terminal is connected to a microphone 31, a speaker 32, a camera 33, a display device 34, and an infrared light 35 disposed in the same site. The microphone 31 converts sound into sound data. The speaker 32 generates sound based on the sound data. The camera 33 captures a moving image. The display device 34 displays an image. Although details will be described later, the infrared light 35 is used to detect the user's line-of-sight direction.

  Next, the electrical configuration of the video conference terminal 1 will be described with reference to FIG. Of the electrical configuration of the PC 2, the configuration necessary for describing the present embodiment is the same as the electrical configuration of the video conference terminal 1. The operations performed by the video conference terminal 1 and the PC 2 to execute the video conference, the configuration of stored data, and the like are the same. Therefore, in the following description, the electrical configuration of the PC 2 is assigned the same number as the configuration of the video conference terminal 1, and this detailed description is omitted. A description of operations performed by the PC 2 is also omitted.

  As shown in FIG. 2, the video conference terminal 1 includes a CPU 10 that controls the video conference terminal 1. A ROM 11, a RAM 12, a hard disk drive (hereinafter referred to as “HDD”) 13, and an input / output interface 19 are connected to the CPU 10 via a bus 18.

  The ROM 11 stores a program for operating the video conference terminal 1, an initial value, and the like. The RAM 12 temporarily stores various information used in the control program. The HDD 13 is a non-volatile storage device that stores various types of information. Instead of the HDD 13, a storage device such as an EEPROM or a memory card may be used.

  Connected to the input / output interface 19 are an audio input processing unit 21, an audio output processing unit 22, a video input processing unit 23, a video output processing unit 24, an operation unit 25, an external communication I / F 26, and an infrared light 35. . The voice input processing unit 21 processes voice data input from the microphone 31. The audio output processing unit 22 processes the operation of the speaker 32. The video input processing unit 23 processes input of image data (video data) from the camera 33. The video output processing unit 24 processes the operation of the display device 34. The operation unit 25 is used for the user to input various instructions to the video conference terminal 1. The external communication I / F 26 connects the video conference terminal 1 to the network 8.

  Next, with reference to FIG. 3, the outline | summary of the operation | movement which the video conference terminal 1 performs is demonstrated. As shown in FIG. 3, the video conference terminal 1 receives image data from another communication terminal connected via the network 8. The video conference terminal 1 displays an image on the display device 34 based on the received image data. FIG. 3 shows an image that the video conference terminal 1 at the base A displays on the display device 34 when the video conference is executed by five communication terminals arranged at the bases A, B, C, D, and E, respectively. An example is shown. During the video conference, the video conference terminal 1 forms the same number of display areas 41 to 44 in the display device 34 as the other communication terminals. Images of other bases are displayed in the display areas 41 to 44, respectively. In addition, the video conference terminal 1 generates sound from the speaker 32 based on the sound data received from another communication terminal. As a result, a plurality of users at different bases can perform a remote conference.

  In addition, the video conference terminal 1 detects that a conversation has occurred between users of two communication terminals among the plurality of communication terminals. Furthermore, the communication terminal of the user who is in conversation (hereinafter referred to as “conversation terminal”) is specified from among the plurality of communication terminals. When the video conference terminal 1 detects a conversation, it performs processing for preferentially securing the bandwidth of the network 8 between the two conversation terminals. Specifically, a conversation terminal is a non-conversation terminal (hereinafter referred to as “relay”) that directly transmits voice data and image data (hereinafter referred to as “communication data”) from a plurality of non-conversation terminals that are not engaged in conversation. "Terminal"). The conversation terminal transmits communication data only to the communication terminal of the conversation partner (hereinafter referred to as “partner terminal”) and the selected non-conversation terminal. The relay terminal transfers the communication data received from the conversation terminal to other non-conversation terminals. As a result, the path of communication data transmitted by the conversation terminal is reduced, and the bandwidth of the network 8 of the conversation terminal is secured.

  Here, the video conference terminal 1 can use a method of detecting and using the user's line-of-sight direction and a method of using voice recognition in order to detect the start of conversation and specify the conversation terminal.

  The detection of the user's line-of-sight direction performed by the video conference terminal 1 will be described. The video conference terminal 1 can detect the line-of-sight direction of users in the same base. It is determined which of the plurality of display areas 41 to 44 formed in the display device 34 is in the detected line-of-sight direction. The video conference terminal 1 identifies the conversation partner of the user of the terminal itself as the user at the base where the image is displayed in the display area in the line-of-sight direction.

  The detection of the gaze direction will be specifically described. A well-known method may be used as a method for detecting the line-of-sight direction. For example, the method described in Japanese Patent Application Laid-Open No. 10-108843 can be applied. According to this method, the video conference terminal 1 causes the infrared light 35 to emit infrared light. Infrared rays are reflected by the eyeball cornea reflection surface of the user to form a virtual image (Purkinje image). The camera 33 captures the formed Purkinje image and the pupil center of the eyeball. The video conference terminal 1 can detect the user's line-of-sight direction from the relative position between the Purkinje image and the pupil center. When the user's line-of-sight direction is detected, it is determined whether the display area of the image at another base is in the line-of-sight direction, and the conversation partner is specified. Details of this processing will be described later.

  The voice recognition performed by the video conference terminal 1 will be described. The video conference terminal 1 can perform well-known voice recognition on the input voice data. Based on the recognized voice, the user's conversation partner is specified, and whether or not the conversation has ended is determined.

  Specifically, the HDD 13 stores an acoustic model, a language model, and a word dictionary for performing speech recognition. When the audio conference data is input, the video conference terminal 1 analyzes the input audio data, extracts the feature amount, and then performs matching between the acoustic model and the language model. As a result, the likelihood is obtained for each sentence acceptable by the language model, and the sentence with the highest likelihood is obtained as the recognition result. When matching, the language model refers to a word dictionary. In addition, when the likelihood becomes a value equal to or less than a predetermined threshold, a recognition result is not obtained as a recognition failure. When the video conference terminal 1 recognizes the registered user name of the communication terminal from the start part of the conversation, the video conference terminal 1 determines that the conversation for the recognized user has started and performs the process. In addition, the video conference terminal 1 determines that the conversation has ended when it recognizes a specific voice for determining that the conversation between the users at the two bases has ended, such as “above”, “end”, and “everyone”. To do. Details of this processing will be described later.

  Next, data stored in the HDD 13 of the video conference terminal 1 will be described with reference to FIGS. 4 and 5. The HDD 13 is provided with various storage areas such as a local terminal information storage area 131, a communication partner information storage area 132, and a conversation history storage area 133 (see FIG. 2).

  The own terminal information storage area 131 stores the registered user name, processing capability, and priority of the own terminal. The registered user name is a name of a user registered in advance in order to specify a user who uses the terminal. The user can also register a plurality of registered user names in advance. For example, if the user is called “Suzuki” by a colleague and “director” by a subordinate, both “Suzuki” and “director” can be registered as registered user names. The registered user name is used to specify a conversation partner by voice recognition. The processing capability is a value indicating the data processing capability of the terminal itself. In the present embodiment, the clock frequency of the CPU 10 of the own terminal is stored as the processing capability. However, information that can be used as processing capability is not limited to this. For example, other information such as the memory capacity of the own terminal may be used as a value indicating the processing capability of the own terminal. The priority indicates the priority of the own terminal among a plurality of communication terminals in the video conference system 3. The priority is set in advance by the user when the video conference is executed.

  In the communication partner information storage area 132, various types of information regarding other communication terminals in the video conference system 3 are stored as shown in FIG. Specifically, the registered user name, transmission time, available bandwidth, processing capability, and priority are stored for each terminal. Here, the transmission time is a data transmission time between the own terminal and the communication partner terminal. The transmission time is measured by a known method at the start of the video conference. In the present embodiment, the time is synchronized between the own terminal and the communication partner terminal, the time stamp attached to the data received from the communication partner terminal, the time measured by the own terminal, The transmission time is measured by comparing. However, the transmission time may be measured from the time until a response is obtained by transmitting a PING command or the like to the communication partner terminal. The available bandwidth is the bandwidth of the network 8 that can be used with the communication partner terminal, and is a value indicating the speed of data communication. The available bandwidth depends on the network 8 used in the video conference system 3, the connection line for connecting to the network 8, the interface, and the like. In the present embodiment, the available bandwidth with each of the other communication terminals is measured by a known method from the start of the video conference.

  The video conference terminal 1 acquires various information stored in the communication partner information storage area 132 when starting the video conference. Transmission time, available bandwidth, processing capacity, and priority are for selecting a relay terminal that directly transmits voice data and image data from a plurality of non-conversation terminals when the terminal becomes a conversation terminal. Used for. Details of this will be described later.

  In the conversation history storage area 133, as shown in FIG. 5, the history of the partner terminal that is the conversation partner of the user of the terminal itself is stored in order. The example shown in FIG. 5 shows the conversation history of the video conference terminal 1 at the site A. In this example, the previous partner terminal is the base B terminal, the previous partner terminal is the base D terminal, the previous partner terminal is the base B terminal, and the previous partner terminal is the base E terminal. Indicates that there is. The conversation history is also used to select a non-conversation terminal.

  Hereinafter, processing performed by the video conference terminal 1 and the PC 2 according to the present embodiment will be described with reference to FIGS. Since the process performed by the video conference terminal 1 and the PC 2 is the same, the process performed by the video conference terminal 1 will be described below. A main process, a conversation start detection process, and a relay terminal acceptance process described below are executed by the CPU 10 according to a program stored in the ROM 11.

  The main process shown in FIG. 6 is started when an instruction to execute a video conference is input. When the main process is started, a meeting start process is first performed (S1). In the conference start process, transmission / reception of various information for executing the process during the video conference, storage, setting of a processing method, and the like are performed.

  As shown in FIG. 7, in the conference start process, first, the registered user name, processing capability, and priority stored in the own terminal information storage area 131 (see FIG. 2) of the HDD 13 are acquired ( S21). The acquired information of the own terminal is transmitted to other communication terminals in the video conference system 3 (S22). Information of each registered user name, processing capability, and priority of the other communication terminal is received from the other communication terminal and stored in the communication partner information storage area 132 (see FIG. 4) of the HDD 13 (S23).

  The transmission time of data with each of the other communication terminals is measured and stored in the communication partner information storage area 132 (S24). The available bandwidth of the network 8 with each of the other communication terminals is measured and stored in the communication partner information storage area 132 (S25). The available bandwidth is appropriately measured during the video conference and updated each time.

  Next, a relay terminal selection method is set (S26). As described above, the relay terminal is a terminal that receives communication data from a conversation terminal and transfers it to another non-conversation terminal among non-conversation terminals. When the video conference terminal 1 becomes a conversation terminal, the relay terminal is selected using any information of conversation history, data transmission time, available bandwidth, data processing capacity, and priority. Can do. In the process of S26, which information is used to select a relay terminal is set according to the operation of the operation unit 25 (see FIG. 2) by the user.

  Next, the identification method of the partner terminal is set (S27). As described above, the video conference terminal 1 can use a method of detecting the user's line-of-sight direction and a method based on voice recognition in order to specify the partner terminal of the conversation partner. In the process of S27, it is set which method is used to specify the partner terminal. Next, reception of audio data and image data from another communication terminal is started, generation of audio based on the received communication data and display of an image are started, and a video conference is executed (S28). The process returns to the main process.

  Returning to the description of FIG. When the conference start process (S1) is finished and the video conference is started, the conversation start detection process is started (S2). In the conversation start detection process, it is detected that a conversation is started between two communication terminals. Further, two conversation terminals are specified. Here, either the first conversation start detection process (see FIG. 8) or the second conversation start detection process (see FIG. 9) is executed according to the setting made in the conference start process (FIG. 7, S27). Is done. The conversation start detection process is executed in parallel with the main process.

  The first conversation start detection process will be described with reference to FIG. The first conversation start detection process is executed when a setting for specifying a partner terminal using voice recognition is performed. CPU10 acquires the audio | voice data about all the bases including the base of an own terminal, and performs audio | voice recognition with respect to the acquired audio | voice data. First, from the result of speech recognition, it is determined whether or not the registered user name is uttered at the start of utterance at any base (S31). Specifically, it is assumed that the utterance is started when the utterance is performed after the silence state for a predetermined time or longer. The registered user name is acquired from the HDD 13, and it is determined whether or not any of the registered user names of the own terminal and other communication terminals is included in the start part of the utterance. If the registered user name is not included (S31: NO), this determination is repeated. If the registered user name is included (S31: YES), it is determined whether or not the silent state after the registered user name has reached a certain time (0.5 seconds in this embodiment) (S32). . In this embodiment, if the silent time after the registered user name is less than a certain time, the registered user name spoken is the user name that appeared in the middle of a series of conversations, that is, other than the purpose of calling the conversation partner. It is determined that the user name is spoken for the purpose. Therefore, if the silent state is less than 0.5 seconds (S32: NO), the process returns to the determination of S31. If the silent state has reached 0.5 seconds (S32: YES), it is determined that a certain user has made a call to one of the other users, and it is determined that conversation between the users at the two bases has started at that time. Is done.

  If it is determined that the conversation has started, it is determined whether or not the registered user name included in the start part of the utterance is the registered user name of the terminal (S34). If it is not the registered user name of the own terminal (S34: NO), it is determined whether or not the voice data of the utterance including the registered user name is the voice data input to the own terminal (S35).

  When the registered user name of the own terminal is included, the user of the other communication terminal calls for the user of the own terminal. Further, if the registered user name is included in the voice data input to the own terminal, the user of the own terminal calls for a user of another communication terminal. Accordingly, when the registered user name of the own terminal is included (S34: YES), or when the voice data is input to the own terminal (S35: YES), the own terminal is one of the two conversation terminals. (S36). Conversation terminal specifying information indicating that the terminal is a conversation terminal is transmitted to all other communication terminals (S37). The conversation terminal specifying information is received from the other conversation terminal (S38). The communication terminal that is the transmission source of the conversation terminal identification information is identified as the partner terminal that is the terminal of the conversation partner user (S39). The identified partner terminal is added to the “previous” column of the conversation history (see FIG. 5) (S40), and the process returns to the determination of S31.

  If the registered user name included in the utterance is not the registered user name of the own terminal (S34: NO) and the voice data is not the voice data input to the own terminal (S35: NO), the own terminal is a non-conversation terminal (S42). The conversation terminal specifying information is received from the other two communication terminals that have become conversation terminals (S43). Two communication terminals that are the transmission source of the conversation terminal identification information are identified as conversation terminals (S44), and the process returns to the determination in S31.

  The second conversation start detection process will be described with reference to FIG. The second conversation start detection process is executed when the setting for specifying the partner terminal is performed using detection of the user's gaze direction. First, it is determined whether or not input of audio data from the microphone 31 (see FIG. 2) connected to the own terminal is started (S51). When the input is started (S51: YES), a captured image is acquired from the camera 33 (see FIG. 2) connected to the terminal (S52), and the above-described line-of-sight direction detection processing is performed (S53). ). It is determined whether or not any of the display areas 41 to 44 (see FIG. 3) exists in the detected line-of-sight direction of the user (S54). If there is no display area in the line-of-sight direction (S54: NO), the process returns to the determination in S51. If there is any display area in the line-of-sight direction (S54: YES), it is determined that the user of the own terminal has started a conversation with another user, and the own terminal is determined to be one of the conversation terminals ( S56). A communication terminal corresponding to the display area in the line-of-sight direction is identified as the counterpart terminal (S57). The identified partner terminal is stored in the conversation history (see FIG. 5) (S58). Conversation terminal specifying information indicating that the specified partner terminal is a conversation terminal is transmitted to all other communication terminals (S59). The process returns to the determination in S51.

  If the input of voice data from the microphone 31 has not been started (S51: NO), it is determined whether or not the conversation terminal specifying information has been received from another communication terminal (S61). If not received (S61: NO), the process returns to the determination of S51. If received (S61: YES), whether or not the terminal is specified as a conversation terminal is determined based on the received conversation terminal identification information (S63). If the own terminal is specified as the conversation terminal (S63: YES), it is determined that the own terminal is one of the conversation terminals (S64). The communication terminal that is the transmission source of the conversation terminal specifying information is specified as the partner terminal (S65). The partner terminal is stored in the conversation history (S66), and the process returns to the determination in S51.

  If the own terminal is not specified as a conversation terminal (S63: NO), it is determined that the own terminal is a non-conversation terminal (S68). The communication terminal specified as the conversation terminal by the conversation terminal specifying information and the communication terminal as the information transmission source are specified as the conversation terminal (S69). The process returns to the determination in S51.

  Returning to the description of FIG. When the conversation start detection process is started (S2), first, normal communication data is transmitted and received. Specifically, input data from the microphone 31 and the camera 33 in the same base is encoded, and medium-quality audio data and image data are generated. The generated communication data is transmitted to all other communication terminals in the same manner (S3). The medium quality data is data having a data amount between high quality data and low quality data, which will be described later. The amount of data varies depending on the compression rate at the time of encoding, the sampling rate, the image resolution, the frame rate of the image data, the error correction code, and the like. The quality of output audio and image is higher as the amount of data is larger.

  Next, it is determined whether or not the start of the conversation between the users at the two bases has been detected by the above-described conversation start detection process (see FIG. 8 or FIG. 9) (S4). Specifically, in the conversation start detection process, when it is determined that the terminal is a conversation terminal or a non-conversation terminal, it is determined that a conversation start has been detected (S36, S42, S56, S64, S68). If the conversation start is not detected (S4: NO), transmission / reception of normal communication data is continued. When the conversation start is detected (S4: YES), it is determined whether or not the terminal is a conversation terminal (S5). If the own terminal is a conversation terminal (S5: YES), processing as a conversation terminal for securing the bandwidth of the network 8 with the partner terminal is performed. First, relay terminal selection processing for selecting a relay terminal from a plurality of non-conversation terminals is performed (S6). As described above, the relay terminal is a communication terminal in which a conversation terminal directly transmits voice data and image data among non-conversation terminals. The relay terminal transfers the communication data received from the conversation terminal to other non-conversation terminals.

  As shown in FIG. 10, when the relay terminal selection process is started, one corresponding non-conversation terminal is selected as a relay terminal candidate according to the setting (FIG. 7, S26) performed in the conference start process. (S71). Details of the non-conversation terminal selection method will be described later. Next, selection information indicating that the relay terminal candidate has been selected is transmitted to the selected non-conversation terminal (S72). It is determined whether acceptance information has been received from the selected non-conversation terminal (S73). Although details will be described later, the non-conversation terminal selected as a candidate transmits either acceptance information indicating acceptance of selection or rejection information indicating rejection. When the refusal information is received (S73: NO), the selected non-conversation terminal is excluded from the relay terminal candidates (S74), and the relay terminal candidates are selected again from the remaining non-conversation terminals (S71). ). When the acceptance information is received (S73: YES), the non-conversation terminal selected as a candidate is determined as the relay terminal (S75), and the process returns to the main process.

  Next, the relay terminal acceptance process will be described with reference to FIG. The relay terminal acceptance process is executed in parallel with the main process during the video conference. First, it is determined whether or not the conversation between the users at the two bases has ended (S81). In the present embodiment, when the silent state continues for a predetermined time or more, it is determined that the conversation has ended. In addition, when a specific voice for determining that the conversation between the users at the two bases is finished, such as “above”, “end”, “everyone”, the conversation is judged to be finished. If it is determined that the conversation has not ended (S81: NO), it is determined whether the start of a new conversation has been detected (S82). If the start of a new conversation is not detected in the conversation start detection process (see FIGS. 8 and 9) (S82: NO), it is determined whether selection information has been received from another communication terminal (S83). . If not received (S83: NO), the process returns to the determination of S81.

  When the selection information is received (S83: YES), it is determined by the acceptance flag whether or not the own terminal has already been selected as the relay terminal (S84). The in-acceptance flag is set to ON when the terminal is selected as a relay terminal, and is set to OFF when the terminal is not selected. If it is not selected as a relay terminal (S84: NO), the acceptance flag is turned ON (S85). Acceptance information indicating acceptance of selection to the relay terminal is transmitted to the conversation terminal that is the transmission source of the selection information (S86). The process returns to the determination in S81. On the other hand, if the relay terminal is already selected (S84: YES), rejection information indicating that the selection is rejected is transmitted (S87), and the process returns to the determination of S81. When it is determined that the conversation has ended (S81: YES), and when the start of a new conversation is detected (S82: YES), the in-acceptance flag is turned off (S88), and processing is performed. Returns to the determination in S81.

  Here, the details of the method for selecting candidate relay terminals (see FIG. 10, S71) will be described. As described above, the relay terminal is selected using any information of conversation history, data transmission time, available bandwidth, data processing capability, and priority.

  When it is set to select a relay terminal using the conversation history, the conversation history stored in the conversation history storage area 133 (see FIG. 5) of the HDD 13 is referred to. Among the non-conversation terminals, the order stored in the conversation history is selected from the new non-conversation terminals as relay terminal candidates. A non-conversation terminal with a newer order stored in the conversation history generally has a higher relevance to the current conversation. The voice data and the image data are directly transmitted from the conversation terminal to the relay terminal. Therefore, the delay time from when the conversation terminal transmits data to when it receives data is shorter than that of a non-conversation terminal that is not selected as a relay terminal. Therefore, by selecting a non-conversation terminal with the newest order stored in the conversation history as a relay terminal, it is possible to minimize the reception delay of the non-conversation terminal highly related to the current conversation.

  When the relay terminal is set to be selected using the data transmission time, the transmission time stored in the communication partner information storage area 132 (see FIG. 4) of the HDD 13 is referred to. Among the non-conversation terminals, a non-conversation terminal having a short data transmission time with the conversation terminal is selected as a relay terminal candidate. The relay terminal transfers the communication data received from the conversation terminal to another non-conversation terminal. Therefore, if the transmission time between the conversation terminal and the relay terminal is long, the reception delay of all other non-conversation terminals is also increased accordingly. By preferentially selecting a non-conversation terminal with a short transmission time as a relay terminal, the reception delay of the entire non-conversation terminal can be reduced.

  If the relay terminal is set to be selected using the available bandwidth, a non-conversation terminal having a wide available bandwidth (a high bandwidth value) is selected as a candidate for the relay terminal. The wider the available bandwidth, the faster the data communication speed. Therefore, the reception delay of the entire non-conversation terminal can be reduced.

  When the relay terminal is set to be selected using the data processing capacity, the non-conversation terminal having a high processing capacity is selected as a relay terminal candidate. A non-conversation terminal with high data processing capability can perform processing from receiving communication data from a conversation terminal to transferring it to another non-conversation terminal more quickly than a non-conversation terminal with low processing capability. Therefore, the reception delay of the entire non-conversation terminal can be reduced.

  If the relay terminal is set to be selected using the priority, the priority stored in the communication partner information storage area 132 (see FIG. 4) of the HDD 13 is acquired. A non-conversation terminal having a high priority is selected as a relay terminal candidate. In this case, the user can minimize the reception delay of a desired communication terminal by appropriately setting the priority.

  Returning to the description of FIG. When a relay terminal is selected (S6), high-quality audio data / image data and low-quality audio data / image data are generated from the same audio and image (S7). Specifically, input data from the microphone 31 and the camera 33 in the same base are encoded under different conditions, and two types of communication data having different data amounts are generated. Next, the high quality data is transmitted to the partner terminal that is the conversation partner (S8). Low quality data is transmitted to the selected relay terminal (S9). None of the communication data is transmitted to non-conversation terminals other than the relay terminal. Next, it is determined whether or not the start of a new conversation has been detected (S10). If detected (S10: YES), the process returns to the determination in S5. If the start of a new conversation has not been detected (S10: NO), it is determined whether or not the conversation with the user of the partner terminal has ended (S11). This determination condition is the same as the determination condition of S81 in FIG. If the conversation has not ended (S11: NO), the process returns to S7, and transmission of high-quality data and low-quality data is continued. When the conversation ends (S11: YES), the process returns to S3, and normal data transmission / reception, that is, transmission of medium quality data to all other communication terminals is performed again.

  When the conversation is started, if the terminal is not a conversation terminal (S5: NO), processing as a non-conversation terminal is performed. First, low quality data is generated (S12). The generated low quality data is transmitted to all other communication terminals including the conversation terminal (S13). Next, it is determined whether or not the relay terminal is selected (S14). If it is not selected (S14: NO), the process proceeds to the determination of S16 as it is. If the relay terminal is selected (S14: YES), the low quality data received from the conversation terminal is transferred to the other non-conversation terminal, and the low quality data is broadcast in the non-conversation terminal (S15). In other words, the other non-conversation terminals are communication terminals excluding two conversation terminals and one relay terminal (own terminal) among all communication terminals. Here, the broadcast in this embodiment means that the same data is not repeatedly transmitted to the same non-conversation terminal, and the data is not looped in a plurality of non-conversation terminals. Indicates that the low-quality data generated by is transmitted to the non-conversation terminal. In the present embodiment, the low quality data transmitted by the conversation terminal is transferred in one direction from the selected one relay terminal to each of the other non-conversation terminals. As a result, it is possible to prevent the low quality data transmitted by the conversation terminal from being repeatedly transmitted to the same non-conversation terminal, or the data from being looped within the non-conversation terminal.

  Next, it is determined whether or not the start of a new conversation has been detected (S16). If it is detected (S16: YES), the process returns to the determination of S5. If the start of a new conversation has not been detected (S16: NO), it is determined whether the conversation has ended (S17). If the conversation has not ended (S17: NO), the process returns to S12, and the process as a non-conversation terminal is continued. When the conversation ends (S17: YES), the process returns to S3, and normal communication data is transmitted and received.

  Next, with reference to FIG. 12 to FIG. 14, an aspect of transmission / reception of communication data when the video conference terminal 1 and the PC 2 perform the above processing will be described. FIG. 12 to FIG. 14 show an example of transmission / reception of communication data in the video conference system 3 including communication terminals at five sites A, B, C, D, and E. FIG. 12 shows a mode of transmission / reception when a conversation between users at two bases is not performed. FIG. 13 shows a transmission / reception mode when a user at the site A and a user at the site E are having a conversation. In FIG. 14, only the transmission / reception mode of the communication terminal at the site A in the case of FIG. 13 is illustrated.

  As shown in FIG. 12, when a conversation between users at two sites is not performed, each communication terminal transmits medium quality data to all of the other communication terminals. As a result, each communication terminal can output audio and video from all other locations. Therefore, a remote conference using voice and video can be executed. However, the bandwidth of the network 8 (see FIGS. 1 and 2) may vary. For example, when users are talking between two out of five sites, if the bandwidth drops and audio and video are interrupted, the conference does not proceed to the next stage, and the remote conference cannot be performed smoothly. . Therefore, when a conversation is performed between users at two sites, it is desirable to preferentially secure the bandwidth of the network 8 between the communication terminals of the users who are performing the conversation.

  As shown in FIG. 13, in the video conference system 3, when a user at the site A and a user at the site E have a conversation, the conversation terminals that are communication terminals at the sites A and E transmit and receive communication data to and from each other. Since communication data is directly transmitted and received between conversation terminals, the time from when communication data is transmitted from one conversation terminal to when the other receives is short. Therefore, the conversation between the users at the two bases can be performed smoothly. The conversation terminal selects any one of the three non-conversation terminals B, C, and D as a relay terminal, and transmits communication data only to the selected non-conversation terminal.

  As shown in FIG. 14, the communication terminal at site A does not need to transmit communication data to the non-conversation terminals at sites C and D while operating as a conversation terminal. Therefore, when operating as a conversation terminal, the number of communication data transmission / reception paths can be reduced. Therefore, when the communication terminal according to the present embodiment operates as a conversation terminal, the network terminal can be used efficiently without increasing the entire network band that can be used between the communication terminals at the bases A to E. The network bandwidth that can be used for the network can be secured. There is no pressure on the network bandwidth of devices other than the bases A to E. There is no need to go through complicated processing such as changing the network to be used. Then, the voice and image communication data at the site A is transferred from the relay terminal at the site B to the non-conversation terminals at the sites C and D. Therefore, in the video conference system 3, even if the conversation terminal does not transmit communication data to all of the other communication terminals, the remote conference can be executed smoothly. In other words, when a conversation is started between users at two sites, a remote conference using all communication terminals including non-conversation terminals is continuously executed smoothly, and a network bandwidth that can be used by the two conversation terminals is secured. be able to.

  As shown in FIG. 13, the two conversation terminals can select different non-conversation terminals as relay terminals. In this case, the processing load for transferring the communication data can be distributed without being concentrated on one of the plurality of non-conversational terminals.

  The quality of communication data will be described. As shown in FIGS. 13 and 14, communication data transmitted and received between conversation terminals (A, E) is high-quality data with a large data amount. Accordingly, the conversation terminals (A, E) can provide voice and video to a user who is in conversation with high quality. On the other hand, the conversation terminal (A, E) transmits low quality data with a small amount of data to the relay terminal (B or D). The conversation terminal (A, E) secures the bandwidth of the network 8 that can be used with the other terminal more reliably by reducing the amount of communication data transmitted to the relay terminal (B or D). Can do. Furthermore, since the amount of communication data to be broadcast is small, the non-conversation terminals (B, C, D) can smoothly broadcast the communication data transmitted from the conversation terminals (A, E). In addition, when the conversation is started between the two conversation terminals, the non-conversation terminals (B, C, D) set the data to be transmitted as low quality data. As a result, the amount of communication data transmitted to the conversation terminals (A, E) is reduced. Therefore, a bandwidth that can be used by the conversation terminals (A, E) can be secured.

  When each communication terminal detects the end of the conversation between the users at the two locations, it directly transmits medium quality data to all of the other communication terminals (see FIG. 12). Therefore, each communication terminal can proceed with the remote conference while minimizing the data reception delay when no conversation is performed between the users at the two bases. In addition, each conference terminal can easily specify the partner terminal that is the communication terminal of the conversation partner by using at least one of voice recognition and detection of the user's line-of-sight direction.

  In the above embodiment, the video conference terminal 1 and the PC 2 correspond to the “communication terminal” of the present invention. The CPU 10 that detects that a conversation has started between the user of the terminal itself and the user of another communication terminal in S36 of FIG. 8 and S56 and S64 of FIG. 9 functions as “first start detection means”. The CPU 10 that specifies the counterpart terminal in S38 and S39 in FIG. 8 and S57 and S65 in FIG. 9 functions as “first specifying means”. The CPU 10 that performs the relay terminal selection process of FIG. 10 functions as “selection means” and “first to fifth selection means”. The CPU 10 that generates communication data in S3, S7, and S12 of FIG. 6 functions as a “generating unit”. The high quality data of the above embodiment corresponds to “first data” of the present invention. The low quality data corresponds to “second data”, and the medium quality data corresponds to “third data”.

  CPU10 which transmits high quality data to the other party terminal by S8 of FIG. 6 functions as a "first transmission means". CPU10 which transmits low quality data to a relay terminal by S9 of FIG. 6 functions as a "2nd transmission means." The CPU 10 that detects the start of conversation between the other two communication terminals in S42 of FIG. 8 and S68 of FIG. 9 functions as “second start detection means”. The CPU 10 that specifies the two conversation terminals in S44 of FIG. 8 and S69 of FIG. 9 functions as the “second specifying means”. The CPU 10 that broadcasts the low-quality data between the non-conversation terminals in S15 of FIG. 6 functions as a “transfer unit”.

  The conversation history storage area 133 of the HDD 13 corresponds to “first storage means”. The CPU 10 that stores the counterpart terminal as a history in S40 of FIG. 8 and S58 and S66 of FIG. 9 functions as “storage control means”. The communication partner information storage area 132 of the HDD 13 corresponds to “second storage means” and “third storage means”. In S <b> 71 of FIG. 10, the CPU 10 that acquires the priority from the HDD 13 functions as a “first acquisition unit” when the relay terminal is set using the priority. The CPU 10 that acquires the registered user name from the HDD 13 in S31 of FIG. 8 functions as a “second acquisition unit”. The CPU 10 that acquires the utterance voice of each communication terminal functions as a “third acquisition unit”, and the CPU 10 that performs voice recognition on the acquired utterance voice functions as a “voice recognition unit”. In S39 of FIG. 8, the CPU 10 that specifies the partner terminal using the registered user name included in the input voice functions as a “voice specification unit”.

  The CPU 10 that causes the display device 34 to display an image based on image data received from another communication terminal functions as a “display control unit”. The CPU 10 that acquires the captured video of the terminal in S52 of FIG. 9 functions as a “fourth acquisition unit”. The CPU 10 that detects the user's line-of-sight direction in S53 of FIG. 9 functions as “line-of-sight detection means”. In S54 and S57 in FIG. 9, the CPU 10 that identifies the communication terminal corresponding to the display area in the user's line-of-sight direction as the counterpart terminal functions as “line-of-sight identifying means”. The CPU 10 that detects the end of the conversation in S11 and S17 in FIG. 6 functions as an “end detection means”. The CPU 10 that transmits the medium quality data to all of the other communication terminals in S3 of FIG. 6 functions as a “third transmission unit”.

  The process of detecting that the conversation has started between the user of the own terminal and the user of another communication terminal in S36 of FIG. 8 and S56 and S64 of FIG. 9 corresponds to the “first start detection step”. The process of specifying the partner terminal in S38 and S39 of FIG. 8 and S57 and S65 of FIG. 9 corresponds to the “first specifying step”. The relay terminal selection process in FIG. 10 corresponds to “selection step” and “first to fifth selection steps”. The process of generating communication data in S3, S7, and S12 of FIG. 6 corresponds to a “generation step”. The process of transmitting the high quality data to the counterpart terminal in S8 of FIG. 6 corresponds to the “first transmission step”. The process of transmitting the low quality data to the relay terminal in S9 of FIG. 6 corresponds to the “second transmission step”. The process of detecting the start of conversation between the other two communication terminals in S42 of FIG. 8 and S68 of FIG. 9 corresponds to the “second start detection step”. The process of specifying two conversation terminals in S44 of FIG. 8 and S69 of FIG. 9 corresponds to a “second specifying step”. The process of broadcasting low quality data between non-conversation terminals in S15 of FIG. 6 corresponds to a “transfer step”.

  Needless to say, the present invention is not limited to the above-described embodiment, and various modifications are possible. For example, the number of communication terminals used for the video conference system 3 is not limited to five. When the conversation terminal selects one relay terminal, the present invention can be applied by using four or more communication terminals.

  In the above embodiment, each of the two conversation terminals selects one relay terminal from among the non-conversation terminals. However, one conversation terminal may select two or more relay terminals. Even when two or more relay terminals are selected, the communication data transmission path of the conversation terminal can be reduced. In this case, the relay terminal selected from one conversation terminal may transfer the communication data to a non-conversation terminal that has not yet received communication data from the one conversation terminal.

  The method of broadcasting communication data in the non-conversation terminal is not limited to the method used in the above embodiment. Broadcast may be performed using a well-known method that does not cause duplication of data, loops, or the like, such as a method of transmitting communication data by designating a special address in the network 8. In the example shown in FIG. 14, the relay terminal at the base B directly transfers the communication data to the non-conversation terminals at the bases C and D. However, the communication data may be sequentially transferred from the base B to the base C and further from the base C to the base D.

  In the above-described embodiment, the bandwidth of the network 8 that can be used by the conversation terminal is more reliably secured by controlling transmission / reception of three communication data of high quality, medium quality, and low quality. However, the quality of communication data to be transmitted can be changed as appropriate. For example, communication data of the same quality may be transmitted to each communication terminal regardless of whether the transmission destination of the communication data is a conversation terminal. According to the present invention, since the communication data transmission path of the conversation terminal can be reduced without changing the quality of the communication data, a band that can be used by the conversation terminal can be secured. Similarly, non-conversational terminals may transmit communication data of the same quality regardless of whether or not a conversation is being performed.

  In the above embodiment, the two conversation terminals select different non-conversation terminals as relay terminals. As a result, the processing load for transferring communication data can be distributed to the two non-conversation terminals. However, the non-conversation terminal in which two conversation terminals are the same may be selected as a relay terminal. For example, the two conversation terminals both select the non-conversation terminal having the highest processing capability as the relay terminal. When communication data transmitted from two conversation terminals is transferred to another non-conversation terminal via the same relay terminal, there is no difference in reception delay between the two communication data in the non-conversation terminal. . Therefore, the non-conversation terminal can receive communication data from two conversation terminals simultaneously.

  In the above embodiment, each communication terminal specifies a partner terminal using voice recognition and detection of a user's line-of-sight direction. However, the identification method of the counterpart terminal can be changed as appropriate. For example, a user who starts a conversation may operate the operation unit 25 or the like to designate a partner terminal. Alternatively, the amount of audio data transmitted by each communication terminal may be monitored within a certain time, and processing may be performed using two communication terminals having a large amount of data to be transmitted as conversation terminals. Similarly, a determination method for determining whether or not the conversation has ended, a detection method for detecting whether or not the conversation has started, and the like can be changed.

1 Video conference terminal 2 PC
3 Video conference system 10 CPU
13 HDD
34 Display devices 41 to 44 Display area 132 Communication partner information storage area 133 Conversation history storage area

Claims (10)

A communication terminal that is connected to three or more other communication terminals via a network and transmits / receives communication data including at least one of audio data and image data,
A first start detecting means for detecting that a conversation is started between the user of the own terminal and the user of one communication terminal among the other communication terminals;
A first specifying means for specifying a partner terminal that is the one communication terminal on the partner side of the started conversation when the first start detecting means detects that the conversation has started;
Selecting means for selecting at least one non-conversational terminal excluding the counterpart terminal having a conversation among the other communication terminals when the counterpart terminal is specified by the first specifying means;
Generating means for generating communication data from information including at least one of audio data and image data;
First transmission means for transmitting first data which is communication data generated by the generation means to the counterpart terminal specified by the first specification means;
Second transmission means for transmitting, to the non-conversation terminal selected by the selection means, second data which is communication data in which the information generated by the generation means is common to the first data;
Second start detecting means for detecting that a conversation has started between users of two communication terminals in the other communication terminals;
A second specifying means for specifying the two communication terminals performing the started conversation when the second start detecting means detects that the conversation is started;
Transfer means for transferring the received second data to a communication terminal other than the two communication terminals specified by the second specifying means when receiving the second data from the other communication terminal. A communication terminal characterized by that.   2. The communication terminal according to claim 1, wherein the first transmission unit transmits first data having a data amount larger than second data transmitted by the second transmission unit to the counterpart terminal. A storage control means for storing the counterpart terminal specified by the first specification means in the first storage means in order as a history;
The said selection means is provided with the 1st selection means which preferentially selects the non-conversation terminal with the new order which was memorize | stored as a log | history in said 1st memory | storage means among several said non-conversation terminals. Item 3. The communication terminal according to Item 1 or 2. The selecting means is
A second selection means for preferentially selecting a non-conversation terminal having a short transmission time of communication data with the own terminal among the plurality of non-conversation terminals;
A third selection means for preferentially selecting a non-conversation terminal having a high network bandwidth with the own terminal among the plurality of non-conversation terminals;
4. The apparatus according to claim 1, further comprising at least one of a fourth selection unit that preferentially selects a non-conversation terminal having a high data processing capability among the plurality of non-conversation terminals. The communication terminal described. A plurality of communication terminals connected via the network, further comprising first acquisition means for acquiring the priority from second storage means for storing the priority of each communication terminal;
The said selection means is provided with the 5th selection means which preferentially selects the non-conversation terminal with a high priority acquired by said 1st acquisition means among the said non-conversation terminals. The communication terminal according to any one of 1 to 4. Second acquisition means for acquiring the specific information from third storage means for storing the specific information for specifying the user using each of the communication terminals in association with the communication terminal used by the user;
Third acquisition means for acquiring speech of a user who uses each of the plurality of communication terminals;
Voice recognition means for recognizing the utterance content from the utterance voice acquired by the third acquisition means,
The first specifying means includes voice specifying means for specifying the counterpart terminal using the specifying information when the specific information is included in the utterance content recognized by the voice recognizing means. The communication terminal according to any one of claims 1 to 5. Display control means for displaying a user of the other communication terminal on a display means for displaying an image based on communication data transmitted from the other communication terminal;
A fourth acquisition means for acquiring a captured image of the user from an imaging means for imaging the user;
Line-of-sight detection means for detecting the line-of-sight direction of the user from the captured video acquired by the fourth acquisition means,
The first identification unit identifies a communication terminal used by the user in the line-of-sight direction as the partner terminal when the user displayed on the display unit is in the line-of-sight direction detected by the line-of-sight detection unit. The communication terminal according to claim 1, further comprising a specifying unit. End detection means for detecting whether or not the conversation between the users of the two communication terminals has ended after the first start detection means or the second start detection means has detected that the conversation has started.
Third transmission means for transmitting third data, which is communication data generated by the generation means, to all of the other communication terminals when the end detection means detects that the conversation has ended. The communication terminal according to claim 1, further comprising: A communication method performed in a communication system including four or more communication terminals connected to each other via a network and transmitting / receiving communication data including at least one of audio data and image data,
A first start detection step of detecting that a conversation has started between the user of the terminal and the user of one communication terminal among other communication terminals connected to the terminal;
A first specifying step of specifying a partner terminal that is the one communication terminal on the partner side of the started conversation when it is detected that the conversation is started by the first start detecting step;
A selection step of selecting at least one non-conversation terminal excluding the counterpart terminal having a conversation among the other communication terminals when the counterpart terminal is identified by the first identification step;
A generation step of generating communication data from information including at least one of audio data and image data;
A first transmission step of transmitting first data which is communication data generated by the generation step to the counterpart terminal specified by the first specification step;
A second transmission step of transmitting, to the non-conversational terminal selected in the selection step, second data, which is communication data common to the first data, as information that is generated by the generation step;
A second start detection step for detecting that a conversation has started between users of two communication terminals in the other communication terminals;
A second specifying step of specifying the two communication terminals performing the started conversation when it is detected by the second start detecting step that a conversation is started;
A transfer step of transferring the received second data to a communication terminal other than the two communication terminals specified in the second specifying step when receiving the second data from the other communication terminal. A communication method characterized by the above. A communication system comprising four or more communication terminals that transmit and receive communication data including at least one of audio data and image data, wherein a plurality of the communication terminals are connected to each other via a network,
The communication terminal is
A first start detecting means for detecting that a conversation is started between the user of the own terminal and the user of one communication terminal among the other communication terminals;
A first specifying means for specifying a partner terminal that is the one communication terminal on the partner side of the started conversation when the first start detecting means detects that the conversation has started;
Selecting means for selecting at least one non-conversational terminal excluding the counterpart terminal having a conversation among the other communication terminals when the counterpart terminal is specified by the first specifying means;
Generating means for generating communication data from information including at least one of audio data and image data;
First transmission means for transmitting first data which is communication data generated by the generation means to the counterpart terminal specified by the first specification means;
Second transmission means for transmitting, to the non-conversation terminal selected by the selection means, second data which is communication data in which the information generated by the generation means is common to the first data;
Second start detecting means for detecting that a conversation has started between users of two communication terminals in the other communication terminals;
A second specifying means for specifying the two communication terminals performing the started conversation when the second start detecting means detects that the conversation is started;
Transfer means for transferring the received second data to a communication terminal other than the two communication terminals specified by the second specifying means when receiving the second data from the other communication terminal. A communication system characterized by the above.

Images