JP2007288255A - Communication system, information management apparatus, information processing apparatus, information processing method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology capable of easily particularizing users during utterance from users not making statements even when a plurality of users make statements at the same time. <P>SOLUTION: A user terminal 1-1 detects users whose utterance is started and users whose utterance is finished on the basis of voice data or the like transmitted from user terminals 1-2, 1-3 being participants' terminals taking part in a conference. When the configuration of users making statements at the same time is changed, the user terminal 1-1 transmits control information including identification information for identifying the users making statements to the user terminals 1-2, 1-3. The user terminals 1-2, 1-3 particularize the users making statements on the basis of the control information and display information of the users making statements and information of the users not making statements in different forms. The technology above can be applied to apparatuses capable of making voice conversation by the VoIP. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a communication system, an information management apparatus, an information processing apparatus, an information processing method, and a program, and in particular, even when a plurality of users are speaking at the same time, each information processing apparatus A communication system, an information management device, an information processing device, an information processing method, and an information management device, which can easily identify a user who is performing and a user who is not speaking based on information transmitted from the information management device; Regarding the program.

  In recent years, conversation services using VoIP (Voice over IP) using an IP (Internet Protocol) network have begun to spread. By using a VoIP conversation service, a user can use a handset (phone) connected to a personal computer, etc., as if using a telephone using a telephone line, and with an acquaintance at a remote location. You can have a voice conversation.

Japanese Patent Application Laid-Open No. 2004-151867 discloses a technique for multiplexing information related to a user on audio data transmitted and received between devices constituting a video conference system, and displaying simultaneously transmitted images on each device based on the information related to the user. It is disclosed. Depending on this technique, the name of the speaker, the image of the face, and the like are displayed by each device.
JP 11-177952 A

  By the way, as a conversation mode by VoIP, there is a conference mode in which three or more users can participate in the conversation at the same time. In the conference mode, the user's terminal or server that hosts the conference synthesizes the audio captured by the user's terminal participating in the conference, and sends the synthesized audio to each terminal. By outputting, conversation is realized among a plurality of users.

  If a conversation is taking place in such a conference mode, there is no problem because it is possible to identify each other when it is a P-to-P (peer-to-pee) conversation, but three or more users When there is a conversation by multiple users, and when a user speaks at the same time, the VoIP conversation service may not be able to see the face of the other party who is having a conversation like a video conference. , It may be difficult to understand which user it was performed by. A method for solving this problem is not disclosed in Patent Document 1 described above.

  The present invention has been made in view of such a situation, and even when a plurality of users are speaking at the same time, each information processing apparatus is speaking with the user who is speaking. This makes it possible to easily identify a user who is not present based on information transmitted from the information management apparatus.

  The communication system according to the first aspect of the present invention includes an information management device and a plurality of information processing devices connected via a network. Of these, the information management apparatus synthesizes the voices acquired and transmitted by the predetermined information processing apparatus, and transmits the synthesized voices to the plurality of information processing apparatuses for output. Synthesizing means for carrying out conversations between a plurality of users of the information processing devices, detection means for detecting the start or end of the user's speech based on the state of the voice acquired by the plurality of information processing devices; Each time the detection means detects the start or end of a user's speech, and the configuration of the user who speaks at the same time is changed, the first identification information for identifying the users of the plurality of information processing devices and the speech are performed. Generating information for generating control information including second identification information for identifying a user and transmitting the control information to a plurality of the information processing devices, wherein each of the plurality of information processing devices includes the information An acquisition unit that acquires the control information transmitted from the physical device, and each time the control information is acquired by the acquisition unit, does not speak with the user who is speaking based on the control information Display control means for identifying users and displaying information related to each user in different formats.

  An information management device according to a second aspect of the present invention is an information management device connected to a plurality of information processing devices via a network, and synthesizes voices acquired and transmitted by the predetermined information processing device, The speech obtained by the synthesis is acquired by the plurality of information processing devices, and the synthesizing means for causing the plurality of information processing devices to have a conversation by transmitting and outputting the voices to the plurality of information processing devices. Detecting means for detecting the start or end of the user's speech based on the state of the voice, and the detection means detects the start or end of the user's speech, and each time the configuration of the user who speaks is changed Control information including first identification information for identifying a user of the information processing apparatus and second identification information for identifying a user who is making a statement, And a generation means for transmitting device.

  The control information may be an RTCP packet, and the first identification information may be a CSRC described in the RTCP packet.

  The second identification information may be information described in an SDES type RTCP packet as SDES ITEM.

  An information processing method or program according to a second aspect of the present invention synthesizes voices acquired and transmitted by a predetermined information processing apparatus, and transmits the synthesized voices to the plurality of information processing apparatuses. The user of the plurality of information processing devices to have a conversation with each other, detect the start or end of the user's speech based on the state of the voice acquired by the plurality of information processing devices, and The first identification information for identifying the users of the plurality of information processing devices and the user who is speaking Generating control information including two pieces of identification information and transmitting the control information to a plurality of the information processing apparatuses.

  An information processing apparatus according to a third aspect of the present invention includes an acquisition unit that acquires control information transmitted from an information management apparatus, and each time the control information is acquired by the acquisition unit, based on the control information. Display control means for specifying a user who is speaking and a user who is not speaking and displaying information related to each user in different formats.

  An information processing method or program according to a third aspect of the present invention acquires control information transmitted from an information management device, and makes a remark based on the control information each time the control information is acquired. Identifying a user who is speaking and a user who is not speaking, and displaying information related to each user in different formats.

  In the first aspect of the present invention, by synthesizing voices acquired and transmitted by a predetermined information processing apparatus, and transmitting the synthesized voices to the plurality of information processing apparatuses for output. A conversation is performed between a plurality of users of the information processing apparatus. Based on the state of voice acquired by the plurality of information processing devices, the start or end of the user's speech is detected, and each time the configuration of the user who speaks changes, the user of the plurality of information processing devices is identified. Control information including first identification information to be performed and second identification information for identifying a user who is making a statement is generated and transmitted to the plurality of information processing apparatuses.

  In the first aspect of the present invention, the control information transmitted from the information management apparatus is acquired, and a user who makes a statement based on the control information every time the control information is acquired Are identified, and information about each user is displayed in a different format.

  In the second aspect of the present invention, the voice acquired and transmitted by the predetermined information processing apparatus is synthesized, and the synthesized voice is transmitted to the plurality of information processing apparatuses for output. Thus, a conversation is performed among a plurality of users of the information processing apparatus. Based on the state of voice acquired by the plurality of information processing devices, the start or end of the user's speech is detected, and each time the configuration of the user who speaks changes, the user of the plurality of information processing devices is identified. Control information including first identification information to be performed and second identification information for identifying a user who is making a statement is generated and transmitted to the plurality of information processing apparatuses.

  In the third aspect of the present invention, the control information transmitted from the information management device is acquired, and each time the control information is acquired, the user who makes a statement is remarked based on the control information. Users who have not gone are identified, and information about each user is displayed in a different format.

  According to the present invention, even when a plurality of users are speaking at the same time, each information processing apparatus transmits a user who is speaking and a user who is not speaking from the information management apparatus. It can be easily identified based on the information that comes.

  Embodiments of the present invention will be described below. Correspondences between the constituent elements of the present invention and the embodiments described in the specification or the drawings are exemplified as follows. This description is intended to confirm that the embodiments supporting the present invention are described in the specification or the drawings. Therefore, even if there is an embodiment which is described in the specification or the drawings but is not described here as an embodiment corresponding to the constituent elements of the present invention, that is not the case. It does not mean that the form does not correspond to the constituent requirements. On the contrary, even if an embodiment is described herein as corresponding to the invention, this does not mean that the embodiment does not correspond to other than the configuration requirements. .

  The communication system according to the first aspect of the present invention includes an information management apparatus (for example, the user terminal 1-1 in FIG. 1) and a plurality of information processing apparatuses (for example, the user terminal 1 in FIG. 1) connected via a network. -2 and 1-3). Among these, the information management device has substantially the same configuration as the information management device of the second aspect of the present invention, and the information processing device is substantially the same as the information processing device of the third aspect of the present invention. It has the same configuration.

  An information management apparatus (for example, user terminal 1-1 in FIG. 1) according to the second aspect of the present invention is connected to a plurality of information processing apparatuses (for example, user terminals 1-2 and 1-3 in FIG. 1) via a network. The information management devices connected to each other by synthesizing the voices acquired and transmitted by the predetermined information processing device, and transmitting the synthesized voices to the plurality of information processing devices for output. A synthesizing unit (for example, the voice synthesizing unit 32 in FIG. 10) that allows conversation between the users of the plurality of information processing apparatuses, and a user's remark based on the voice status acquired by the plurality of information processing apparatuses Detecting means (for example, the packet analyzing unit 31 in FIG. 10) for detecting the start or end of the user, and the detecting means detects the start or end of the user's speech, and changes the configuration of the user who speaks simultaneously. And generating control information including first identification information for identifying a plurality of users of the information processing devices and second identification information for identifying a user making a statement, and transmitting the control information to the plurality of information processing devices. Generation means (for example, the packet generation unit 33 in FIG. 10).

  An information processing method or program according to a second aspect of the present invention synthesizes voices acquired and transmitted by a predetermined information processing apparatus, and transmits the synthesized voices to the plurality of information processing apparatuses. The user of the plurality of information processing devices to have a conversation with each other, detect the start or end of the user's speech based on the state of the voice acquired by the plurality of information processing devices, and The first identification information for identifying the users of the plurality of information processing devices and the user who is speaking 2 including the step of generating control information including two pieces of identification information and transmitting the control information to the plurality of information processing apparatuses (for example, step S40 in FIG. 12).

  The information processing apparatus (for example, the user terminal 1-2 in FIG. 1) according to the third aspect of the present invention acquires acquisition means (for example, the packet analysis unit 41 in FIG. 11) that acquires control information transmitted from the information management apparatus. ) And each time the control information is acquired by the acquisition means, the user who is making a statement and the user who is not making a statement are identified based on the control information, and the information about each user is expressed in a different format. Display control means (for example, the display control unit 16 in FIG. 9) to be displayed.

  An information processing method or program according to a third aspect of the present invention acquires the control information transmitted from the information management device, and makes a remark based on the control information each time the control information is acquired. This includes a step of identifying a user who is performing and a user who is not speaking and displaying information related to each user in different formats (for example, step S54 in FIG. 13).

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration example of a communication system according to an embodiment of the present invention.

  As shown in FIG. 1, this communication system is configured by connecting user terminals 1-1 to 1-3 to each other via a network 2 such as the Internet. Four or more user terminals may be connected to the network 2. In the example of FIG. 1, the server 3 is also connected to the network 2.

  In the communication system of FIG. 1 having such a configuration, for example, the user terminal (host) that hosts the user terminal 1-1 and the user terminals 1-2 and 1-3 that participate in the conference (users) As a participant's terminal, conference mode voice conversation using VoIP is performed between the users of the user terminals 1-1 to 1-3. Hereinafter, the user of the user terminal 1-1 will be described as user A, the user of the user terminal 1-2 as user B, and the user of the user terminal 1-3 as user C, as appropriate.

  As will be described in detail later, each of the user terminals 1-1 to 1-3 is provided with a microphone that captures the user's voice and a speaker that outputs sound based on data transmitted via the network 2. ing.

As shown in FIG. 1, the user terminal 1-1 transmits voice data A _B that is voice data of the user B captured by the user terminal 1-2 or is transmitted from the user terminal 1-2. When the voice data A _C that is the voice data of the user C captured by 1-3 is transmitted from the user terminal 1-3, the voice data and the voice data of the user A captured by the user C itself And the voice data A _mix obtained by the synthesis is transmitted to the user terminals 1-2 and 1-3.

Further, the user terminal 1-1 not only transmits the audio data A _mix to the user terminals 1-2 and 1-3, but also outputs audio based on the audio data A _{mix and the} like. Listen to remarks, user C's remarks, etc.

The user terminal 1-2 captures the voice of the user B and transmits the obtained voice data _AB to the user terminal 1-1. Further, when the voice data A _mix is transmitted from the user terminal 1-1, the user terminal 1-2 outputs voice based on the voice data A _mix , and the user B speaks to the user B or the user C Listen to remarks.

Similarly, the user terminal 1-3 takes in the voice of the user C and transmits the obtained voice data _AC to the user terminal 1-1. In addition, when the voice data A _mix is transmitted from the user terminal 1-1, the user terminal 1-3 outputs a voice based on the voice data A _mix , and the user C speaks to the user C or the user B Listen to remarks.

  Here, transmission / reception of audio data is performed according to RTP (Real-time Transport Protocol). RTP is a protocol intended to transmit video and audio in real time. The RTP packet includes a time stamp and information indicating the reproduction order.

  In the communication system of FIG. 1, the user terminal 1-1 detects a user who newly participates in a conference or a user who leaves the conference.

When a user who newly participates in the conference or a user who leaves the conference is detected, the user terminal 1-1 updates the identification information of the user participating in the conference registered and managed in the list, and the updated user _Is included in information such as audio data A _mix transmitted to the user terminals 1-2 and 1-3, and transmitted to the user terminals 1-2 and 1-3.

  Furthermore, the user terminal 1-1 is based on the voice data transmitted from the user terminals 1-2 and 1-3, which are the participants' terminals, and the voice data of the user A captured by the user terminal 1-1. Detects the start and end of utterances.

  When the user terminal 1-1 detects that the configuration of the user who speaks at the same time is changed, the user terminal 1-1 transmits control information including identification information for identifying the user who is speaking to the user terminals 1-2 and 1-3. To do.

  For example, when the user B and the user C speak at the same time because the user B starts speaking, the control information transmitted from the user terminal 1-1 at that time includes the user who is speaking. B identification information and user C identification information are included. The user terminals 1-2 and 1-3 that have received the control information can specify from the control information that the user B and the user C are speaking.

  When it is determined that the user B and the user C are speaking, the user terminals 1-2 and 1-3, for example, “user B” and “user C”, which are the names of the users who are speaking, are used. "Is displayed in a different color from" User A ", which is the name of the user who is not speaking, such information is displayed in a different format. A similar display is made on the user terminal 1-1. In addition, it is notified from the user terminal 1-1 to the user terminals 1-2 and 1-3 that there are three users A to C participating in the conference.

  Thereby, the users A to C can confirm from the screen display by the user terminal that the user B and the user C are simultaneously speaking.

Thus, since the control information includes the identification information of the user who is speaking at the same time, the user terminal that has acquired it can easily identify the user who is speaking. For example, if the user terminal that has received the voice data A _mix must analyze the voice data A _mix in order to identify the user who is making a speech, it will be a burden on the user terminal. There is no need to do anything.

  Here, the control information is transmitted from the user terminal 1-1 as an RTCP (RTP Control Protocol) packet. RTCP is a protocol for transmitting information about the sender and receiver of an RTP packet.

  There are mainly five types of RTCP packets: SR (Sender Report), RR (Receiver Report), SDES (Source Description), BYE (Goodbye), and APP (Application Specific).

  The SR type RTCP packet is a packet used by the terminal on the voice data transmitting side to teach its own status to the terminal on the receiving side. The RR type RTCP packet is used by the terminal on the voice data receiving side. , A packet used to tell the sending terminal its own status.

  The SDES type RTCP packet is a packet used for transmitting detailed information of voice data, and the BYE type RTCP packet is a packet used for instructing the organizer's terminal to leave the conference. An APP type RTCP packet is a packet used to teach what application is used to hold a conference.

  Of these, SDES type RTCP packets are used as RTCP packets for transmitting control information. Hereinafter, as appropriate, the SDES type RTCP packet is simply referred to as an SDES packet. Details of the data structure of the RTP packet and the RTCP packet will be described later.

  FIG. 2 is a diagram for explaining a flow of data transmission / reception performed between the user terminals 1-1 to 1-3 in FIG.

  In voice data transmission by VoIP, UDP (User Datagram Protocol) is generally used with priority given to preventing delay. At this time, as described above, RTP is used for transmitting media (voice data), and RTCP is used for controlling media transmission. The terminal on the transmission side of the audio data transmits the target data with an RTP header attached, and the playback procedure in the terminal on the reception side is defined by the description of the RTP header. The sending terminal and receiving terminal regularly exchange RTCP packets, the receiving terminal operates according to RTCP-SR (SR type RTCP packet), and the sending terminal uses RTCP-RR (RR type). Flow control according to the RTCP packet).

When incorporating the user's voice B, in step S11, the user terminal 1-2 transmits the resulting audio data A _B by RTP packets to the user terminal 1-1. In step S12, the user terminal 1-2 transmits an RTCP-RR packet (RR type RTCP packet) to the user terminal 1-1.

  The user terminal 1-1 receives the RTP packet transmitted from the user terminal 1-2 in step S1, and receives the RTCP-RR packet in step S2.

On the other hand, when the user C's voice is captured, in step S21, the user terminal 1-3 transmits the obtained voice data A _C to the user terminal 1-1 using an RTP packet. In step S22, the user terminal 1-3 transmits an RTCP-RR packet to the user terminal 1-1.

  The user terminal 1-1 receives the RTP packet transmitted from the user terminal 1-3 in step S3, and receives the RTCP-RR packet in step S4.

In step S5, if the user terminal 1-1 captures the voice of the user A at that time, the voice data stored in the RTP packet transmitted from the user terminal 1-2 includes the voice of the user A. to synthesize speech data a _C stored in the a _B and RTP packets transmitted from the user terminal 1-3.

In step S6, the user terminal 1-1 transmits the voice data A _mix obtained by the synthesis to the user terminals 1-2 and 1-3 by the RTP packet.

  In addition, the configuration of the user terminal 1-1 is changed so that the user B and the user C speak at the same time, but the user C and the user C speak at the same time. If it is detected, the SDES packet is transmitted to the user terminals 1-2 and 1-3 in step S7. In the SDES packet transmitted here, the identification information of each of the users A to C participating in the conference and the identification of each of the users B and C, which are the main voices included in the RTP packet transmitted in step S6, for example. Contains information.

  The user terminal 1-2 receives the RTP packet transmitted from the user terminal 1-1 in step S13, and receives the SDES packet in step S14.

  In step S15, the user terminal 1-2 outputs voice based on the received RTP packet.

  In step S16, the user terminal 1-2 displays the information of the user who is speaking in a format different from the information of the user who is not speaking based on the information described in the received SDES packet. To do.

  Similarly, the user terminal 1-3 receives the RTP packet transmitted from the user terminal 1-1 in step S23, and receives the SDES packet in step S24.

  In step S25, the user terminal 1-3 outputs voice based on the received RTP packet.

  Further, in step S26, the user terminal 1-3 displays the information of the user who is speaking in a format different from the information of the user who is not speaking based on the information described in the received SDES packet. To do.

  Thus, every time the configuration of a user who speaks at the same time changes, an SDES packet is transmitted from the user terminal 1-1. Further, in the user terminals 1-2 and 1-3, every time an SDES packet is received, the display of the information of the user who is speaking is switched.

  In the above, the conference organizer's device is the user terminal 1-1, but the conference may be managed by the server 3 in FIG. In this case, the user voice data captured by the user terminals 1-1 to 1-3 is transmitted to the server 3 via the network 2, and the server 3 synthesizes them. The synthesized voice is transmitted from the server 3 to the user terminals 1-1 to 1-3, and the voice is output from each user terminal.

  Further, when the server 3 detects the start of the user's speech or the end of the speech, and the configuration of the user who speaks at the same time is changed, control information including identification information of the user who is speaking at the same time is provided. It is transmitted from the server 3 to the user terminals 1-1 to 1-3.

  In the user terminals 1-1 to 1-3, based on the control information transmitted from the server 3, the user who is speaking and the user who is not speaking are specified, and the information of the user who is speaking Is displayed in a format different from the information of the user who is not speaking.

  Here, the data structure of the packet will be described.

  FIG. 3 is a diagram illustrating a data structure of the RTP packet.

  As shown in FIG. 3, an RTP packet is composed of a data link header (Data Link Header), an IP header (IP Header), a UDP header (UDP Header), an RTP header (RTP Header), and a data part (Data). . FIG. 3 shows enlarged data described in the RTP header.

  Of the data described in the RTP header, “V” represents a version number, and “P” represents the presence or absence of padding. For example, when 1 is described as the value of “P”, it indicates that the payload is padded at the end of the packet. “X” is a flag set when there is an extension header immediately after the RTP header.

  “CC” represents the number of identifiers described as CSRC (Contributing Source), and “M” is a marker bit, which represents a boundary between sound and silence in voice data transmission by VoIP.

  “PT” represents a payload type, and “Sequence” represents a packet serial number. With “Sequence” and “Time Stamp” described next, synchronization of reproduction in the terminal on the receiving side that has received the RTP packet is established.

  "SSRC (Synchronization Source)" is a temporary identifier that changes from session to session. In audio data transmission by VoIP, the conference organizer's user terminal controls playback, so the organizer's identifier is assigned. It is done. “CSRC” is an identifier of a participant who uses a user terminal that has prepared voice data included in an RTP packet.

  In the RTP header, “Payload Format Extension” and “Data” are described after “CSRC”.

  The RTP packet has such a structure. For example, the identifier of the user A is described in the SSRC of the RTP packet transmitted from the user terminal 1-2 that is the conference participant's terminal to the user terminal 1-1 that is the organizer's terminal, and the user B is included in the CSRC. Is described. Based on CSRC, the user terminal 1-1 that has received the RTP packet transmitted from the user terminal 1-2 determines that the received RTP packet is transmitted from the user terminal 1-2 used by the user B. It is possible to determine whether or not the user B is speaking from the marker bit value of the RTP packet.

  On the other hand, the identifier of user A is described in the SSRC of the RTP packet transmitted from the user terminal 1-1, which is the conference organizer's terminal, to the user terminal 1-2, which is the participant's terminal, and the user is identified in CSRC. The identifier of B and the identifier of user C are described. Since the CSRC is also rewritten by the user terminal 1-1 each time the conference participant changes, the user terminal of the conference participant can read the conference from the CSRC described in the RTP packet transmitted from the user terminal 1-1. Participants can be confirmed.

  Thus, in the RTP packet transmitted from the user terminal 1-1, since the identifier of the conference participant is described in the RTP header, it is necessary to specify the process for each application of the user terminal that receives the RTP packet. Absent. For example, it is possible to use a data area as an area for describing an identifier of a conference participant, but in this case, an identifier of the participant is described in this part of the data area for each application of the user terminal. It is not necessary to do such a thing.

  FIG. 4 is a diagram showing a data structure of the RTCP packet.

  As shown in FIG. 4, the RTCP packet is composed of a data link header, an IP header, a UDP header, and RTCP data (RTCP Data). The description contents of the RTCP data differ depending on the type of the RTCP packet described above. An SDES packet is used as an RTCP packet transmitted as control information from the user terminal 1-1 which is a conference organizer's terminal.

  FIG. 5 is a diagram illustrating an example of data described in the data portion of the SDES packet.

  “V” represents a version number, and “P” represents the presence or absence of padding. “SC (Source Count)” represents the number of “SSRC / CSRC”, and “PT” represents the payload type. “Packet Length” represents the packet length.

  As shown in FIG. 5, a predetermined number of “SSRC / CSRC” and “SDES ITEM” can be described in the SDES packet. The SDES packet transmitted from the user terminal 1-1, which is the conference organizer's terminal, to the user terminals 1-2, 1-3, which are the participants' terminals, is the same as the number of participants who are speaking at the same time. A number of "SSRC / CSRC" and "SDES ITEM" pairs are described.

  For example, when user B has started speaking and user B and user C are speaking at the same time, the SDES packet transmitted from user terminal 1-1 at that time includes “SSRC / CSRC "And" SDES ITEM "are described only in two sets.

  The first set of SSRC describes the identifier of the user A who is the conference organizer, and the CSRC describes the identifier of the user B who is the conference participant and the identifier of the user C. In the SDES ITEM, incidental information regarding the user B is described.

  The second set of SSRC describes the identifier of the user A who is the conference organizer, and the CSRC describes the identifier of the user B who is the conference participant and the identifier of the user C. In the SDES ITEM, incidental information regarding the user C is described.

  FIG. 6 is a diagram illustrating an example of information described as SDES ITEM.

  As shown in FIG. 6, three types of information “Item Identifier”, “Byte Length”, and “Item Description” are described as SDES ITEM. The definitions of “Item Identifier” and “Item Description” are shown in FIG.

  As shown in FIG. 7, the SDES ITEM identified with an Item Identifier of 0 is “END”, and the Item Description describes information indicating the end of the SDES ITEM.

  The SDES ITEM identified by Item Identifier 1 is “CNAME”, and a unique identifier is described for each participant in Item Description.

  The SDES ITEM whose Item Identifier is 2 is “NAME”, and the name of the participant is described in Item Description.

  The SDES ITEM identified by Item Identifier 3 is “EMAIL”, and the e-mail address of the participant is described in Item Description.

  The SDES ITEM whose Item Identifier is 4 is “PHONE”, and the Item Description describes the phone number of the participant and the phone number of the terminal used by the participant.

  The SDES ITEM identified by Item Identifier 5 is “LOC”, and the address of the participant is described in Item Description.

  The SDES ITEM identified by Item Identifier 6 is “TOOL”, and the name of the application used by the participant is described in Item Description.

  The SDES ITEM identified by Item Identifier 7 is “NOTE”, and Item Description describes information indicating the state of the participant.

  The SDES ITEM identified by Item Identifier 8 is “PRIV”, and the information for application extension is described in Item Description.

  The user terminal 1-1 transmits such an SDES packet in which incidental information of the participant is described to the user terminals 1-2 and 1-3 every time the configuration of the user who speaks at the same time changes. The user terminals 1-2 and 1-3 that have received the SDES packet transmitted from the user terminal 1-1 use information such as CNAME, NAME, EMAIL, PHONE, and LOC described in the SDES packet, respectively. Identify who is currently speaking and display it.

  Since CNAME is the only one that is required to be attached in the description of SDES ITEM, for example, this CNAME is used to notify the information of the user who is having a conversation. In addition, when the terminal used by the user A who has a microphone and a speaker and who has a conversation with the user B and the user C like the user terminal 1-1 hosts the conference, the identifier of the organizer However, when user A is speaking, the identifier of user A may also be described in the SDES packet as the CNAME of the SDES ITEM. That is, CNAME of SDES ITEM represents the user who is making a statement.

  FIG. 8 is a diagram illustrating an example of an identifier described as CNAME of the SDES packet. In FIG. 8, the horizontal axis represents the time axis.

In the example of FIG. 8, the user A starts speaking at time t ₁ and continues until time t ₄ . In addition, the user C starts speaking at time t ₂ and continues until time t ₅ . Furthermore, at time t ₃ , the user B starts speaking and continues until time t ₆ .

Therefore, during time from time t ₁ to time t ₂ , only user A speaks, and during time from time t ₂ to time t ₃ , two users A and C simultaneously speak. It has been broken. Also, during the time from time t ₃ to time t _{4, three} users, user A, user B, and user C, speak at the same time, and during the time from time t ₄ to time t ₅ , user B and user Two of C are speaking at the same time. In the time from time t ₅ to time t ₆ , the speech is made only by user B.

When each user makes a statement at such a timing, as shown in FIG. 8, the SDES packet transmitted from the user terminal 1-1 at time t ₁ includes the user A who started the statement at that time. Contains SDES ITEM whose identifier is described as CNAME.

Further, the SDES packet transmitted from the user terminal 1-1 in time t _2, the added identifier of the user C who start speaking at time t _2, the a SDES ITEM the identifier of the user A is described as CNAME, SDES ITEM in which the identifier of the user C is described as CNAME is included.

The SDES packet transmitted from the user terminal 1-1 at time t _3, further adds the identifier of the user B who start speaking at time t ₃ is, SDES ITEM the identifier of the user A is described as CNAME, the user SDES ITEM in which the identifier of B is described as CNAME and SDES ITEM in which the identifier of user C is described as CNAME are included.

The SDES packet transmitted from the user terminal 1-1 at time t _4, the identifier of the user A has finished speaking at time t ₄ is deleted, and SDES ITEM the identifier of the user B is described as CNAME, user C SDES ITEM in which the identifier is described as CNAME is included.

The SDES packet transmitted from the user terminal 1-1 at time t _5, is deleted identifier of the user C has been completed the speech at time t _5, the identifier of the user B includes SDES ITEM described as CNAME.

At time t ₆ the state had not been anyone speech by the user B ends the remarks, SDES packet from the user terminal 1-1 is not transmitted.

  As described above, the user terminal 1-1 transmits an SDES packet every time the configuration of a user who speaks at the same time changes, and notifies the participant's user terminal whether he / she is speaking.

  A series of operations performed by the user terminal 1-1 which is a conference organizer's terminal and a series of operations performed by the user terminals 1-2 and 1-3 which are conference participant's terminals will be described later with reference to flowcharts. .

  FIG. 9 is a block diagram illustrating a functional configuration example of the user terminal 1-1.

  In the user terminal 1-1, as shown in FIG. 9, a communication control unit 11, a main control unit 12, an input / output control unit 13, a voice input unit 14, a voice output unit 15, and a display control unit 16 are predetermined. This is realized by executing the program. The user terminal 1-1 is provided with a network terminal 21, a microphone 22, a speaker 23, and a display 24.

  The communication control unit 11 is connected to the network 2 via a cable attached to the network terminal 21 and communicates with the user terminals 1-2 and 1-3. When the RTP packet and the RTCP packet are transmitted from the user terminals 1-2 and 1-3, the communication control unit 11 receives them and outputs them to the main control unit 12. Further, when the RTP packet and the RTCP packet (SDES packet) are supplied from the main control unit 12, the communication control unit 11 transmits them to the user terminals 1-2 and 1-3.

  The main control unit 12 combines the audio data included in the RTP packet supplied from the communication control unit 11 and the audio data supplied from the input / output control unit 13, and stores the audio data obtained by the synthesis. Is generated. The main control unit 12 outputs the generated RTP packet to the communication control unit 11. Further, the main control unit 12 outputs audio data included in the RTP packet supplied from the communication control unit 11 to the input / output control unit 13.

  Further, the main control unit 12 makes a statement at the same time based on the status of the RTP packets transmitted from the user terminals 1-2 and 1-3 and the status of the voice data supplied from the input / output control unit 13. When the user configuration is monitored and it is detected that the user configuration has changed, an SDES packet as described above is generated in which the identifier of the user who is speaking is described. The main control unit 12 outputs the generated SDES packet to the communication control unit 11. Note that information indicating who has made a statement and who has not made a statement is also output from the main control unit 12 to the input / output control unit 13.

  The input / output control unit 13 outputs the audio data captured by the audio input unit 14 to the main control unit 12. In addition, the input / output control unit 13 outputs the audio data supplied from the main control unit 12 to the audio output unit 15 to output audio.

  Furthermore, the input / output control unit 13 outputs information supplied from the main control unit 12 to the display control unit 16 indicating who is speaking and who is not speaking.

  The audio input unit 14 performs A / D (Analog / Digital) conversion processing on the audio signal captured by the speaker 22 and outputs the obtained audio data to the input / output control unit 13.

  The audio output unit 15 performs D / A conversion processing and amplification processing on the audio data supplied from the input / output control unit 13, adjusts the sound volume to a predetermined level, and then outputs audio from the speaker 23.

  Based on the information supplied from the input / output control unit 13, the display control unit 16 causes the display 24 to display the information of the user who is speaking, in a format different from the information of the user who is not speaking.

  The user terminals 1-2 and 1-3 have the same configuration as the above configuration. Hereinafter, the configuration of the user terminal 1-1 illustrated in FIG. 9 will be described as being referred to as the configuration of the user terminals 1-2 and 1-3 as appropriate.

  FIG. 10 is a block diagram illustrating a configuration example of the main control unit 12 of FIG.

  As shown in FIG. 10, the main control unit 12 of the user terminal 1-1 that is the conference organizer's terminal includes a packet analysis unit 31, a voice synthesis unit 32, and a packet generation unit 33.

  The packet analysis unit 31 analyzes the RTP packet supplied from the communication control unit 11 and outputs voice data stored in the RTP packet to the input / output control unit 13 and the voice synthesis unit 32. The sound data output to the input / output control unit 13 is used to output sound from the speaker 23.

  The packet analysis unit 31 also makes a statement based on, for example, CSRC and marker bit values described in the RTP packet, and user A's voice data supplied from the input / output control unit 13. Is identified. From the CSRC and the marker bit values described in the RTP packet, it is specified whether or not each of the user B and the user C is speaking. From the voice data of the user A supplied from the input / output control unit 13, It is specified whether or not the user A is speaking.

  When the packet analysis unit 31 identifies the user who is speaking and detects that the configuration of the user who is speaking at the same time is changed, the packet analysis unit 31 outputs the information of the user who is currently speaking to the packet generation unit 33 To do. Information indicating who makes a statement and who is not making a statement is also output from the packet analysis unit 31 to the input / output control unit 13.

  Further, the packet analysis unit 31 analyzes the data received by the communication control unit 11, detects a user who newly participates in the conference or a user who leaves the conference, and the information on those detected users is a packet generation unit. To 33.

  The voice synthesis unit 32 synthesizes the voice data supplied from the packet analysis unit 31 and the voice data supplied from the input / output control unit 13 and outputs the voice data obtained by the synthesis to the packet generation unit 33.

  The packet generator 33 generates an RTP packet that stores the voice data supplied from the voice synthesizer 32, and outputs the generated RTP packet to the communication controller 11.

  Further, the packet generation unit 33 detects that the configuration of the user who makes a statement at the same time is changed, and when the user information is supplied from the packet analysis unit 31, the identifier of the user who makes the statement is the CNAME of the SDES ITEM. Is generated, and the generated SDES packet is output to the communication control unit 11.

  The packet generation unit 33 manages a user list in which information on users participating in the conference is registered, and information on users newly participating in the conference and users leaving the conference is supplied from the packet analysis unit 31. At that time, the registered contents of the user list are updated accordingly. In the user list, identifiers of users who participate in the conference are registered, and this user list is described when CSRC is described in the RTP header of the RTP packet transmitted from the user terminal 1-1 or in the SDES packet. Referenced when describing CSRC.

  FIG. 11 is a block diagram illustrating another configuration example of the main control unit 12 of FIG. FIG. 11 shows the configuration of the main control unit 12 of the user terminal 1-2 which is a conference participant's terminal, for example.

  The main control unit 12 of the user terminal 1-2 includes a packet analysis unit 41 and a packet generation unit 42 as shown in FIG.

  The packet analysis unit 41 acquires the RTP packet transmitted from the user terminal 1-1 and received by the communication control unit 11 of the user terminal 1-2, and the voice data stored in the acquired RTP packet is input / output control unit 13 is output. Based on the audio data output to the input / output control unit 13, audio is output from the speaker 23 of the user terminal 1-2.

  Further, the packet analysis unit 41 determines which user is based on the CSRC of the RTP packet or SDES packet received by the communication control unit 11 of the user terminal 1-2 and the identifier described as CNAME in the SDES ITEM of the SDES packet. The user is speaking, and the user who is not speaking is specified. The packet analysis unit 41 outputs the specified user information to the input / output control unit 13, and displays the information of the user who is speaking in a format different from the information of the user who is not speaking. 24.

  The packet generation unit 42 generates and generates an RTP packet for storing audio data when user B's audio data captured by the microphone 22 is supplied via the audio input unit 14 and the input / output control unit 13. The RTP packet is output to the communication control unit 11. The RTP packet output to the communication control unit 11 is transmitted to the user terminal 1-1.

  Next, the operation of the user terminal having the above configuration will be described with reference to a flowchart.

  First, processing of the user terminal 1-1 that manages the conference will be described with reference to the flowchart of FIG.

  In step S31, the packet analysis unit 31 of the user terminal 1-1 determines whether or not there is a user who has left the conference based on the analysis result of the data received by the communication control unit 11. If it is determined that there is not, the process proceeds to step S32.

  In step S32, the packet analysis unit 31 next determines whether there is a user who has participated in the conference.

  If the packet analysis unit 31 determines in step S32 that there is a user who has participated in the conference, the packet analysis unit 31 outputs information on the user who has newly participated in the conference to the packet generation unit 33, and proceeds to step S33.

  In step S33, based on the information supplied from the packet analysis unit 31, the packet generation unit 33 adds the identifier of the user who has newly joined the conference to the user list. When the user participating in the conference changes, information other than CSRC in the SDES packet is appropriately rewritten.

  On the other hand, if the packet analysis unit 31 determines in step S32 that no user has participated in the conference, the process of step S33 is skipped. After it is determined that no user has joined the conference, or after the identifier of the user who has newly joined the conference is registered in the user list in step S33, the process proceeds to step S35.

  If the packet analysis unit 31 determines in step S31 that there is a user who has left the conference based on the analysis result of the data received by the communication control unit 11, the packet generator 31 generates information on the user who has left the conference. It outputs to the part 33, and progresses to step S34.

  In step S <b> 34, the packet generation unit 33 deletes the identifier of the user who has left the conference from the user list based on the information supplied from the packet analysis unit 31. Thereafter, the process proceeds to step S35.

  In step S35, for example, the packet analysis unit 31 transmits the CSRC and the marker bit values described in the RTP packet transmitted from the user terminal 1-2 or 1-3 and received by the communication control unit 11, and the input / output Based on the voice data of the user A supplied from the control unit 13, it is specified which user is speaking, and it is determined whether there is a user who has ended the speech that has been performed so far. RTP packets and the like are transmitted from the user terminals 1-2 and 1-3 at a predetermined timing, and the voice synthesizer 32 synthesizes voice data stored therein.

  If it is determined in step S35 that there is no user who has finished speaking, the packet analysis unit 31 proceeds to step S36, and then determines whether there is a user who has started speaking.

  In step S36, if the packet analysis unit 31 determines that there is a user who has started speaking, the process proceeds to step S37, and it is determined in step S31 that there is a user who has left the conference, or joined the conference. Since it is determined in step S32 that there is a user, it is determined whether or not it has been confirmed that the configuration of the user participating in the conference has changed.

  In step S37, when the packet analysis unit 31 determines that the configuration of the user participating in the conference has been confirmed, the process proceeds to step S38.

  In step S38, the packet generation unit 33 refers to the user list and rewrites the CSRC of the RTP header of the RTP packet to be transmitted and the CSRC of the RTCP packet.

  On the other hand, in step S37, the packet analysis unit 31 does not determine in step S31 that there is a user who leaves the conference, and does not determine in step S32 that there is a user who has participated in the conference. If it is determined that it has not been confirmed that the configuration of the user participating in the process has changed, the process of step S38 is skipped and the process proceeds to step S39. In this case, the same CSRC as the RTP packet and RTCP packet transmitted so far is described in the RTP header and RTCP packet of the RTP packet.

  In step S39, the packet generation unit 33 adds the identifier of the user who is the principal of the speech determined to have started in step S36 to the SDES ITEM of the SDES packet.

  In step S40, the packet generator 33 adds the RTP header in which CSRC has been rewritten in step S38 to the voice data supplied from the voice synthesizer 32, and sends the obtained RTP packet from the communication controller 11 to the user terminal 1- Send to 2 and 1-3.

  In step S40, the packet generation unit 33 rewrites the CSRC in step S38, and further transmits the SDES packet with the user identifier added in step S39 from the communication control unit 11 to the user terminals 1-2 and 1-3. . Thus, the SDES packet is transmitted from the user terminal 1-1 at the timing when the configuration of the user who speaks at the same time when a certain user starts speaking is changed.

  In the user terminals 1-2 and 1-3, voice is output based on the voice data stored in the RTP packet transmitted here, and a message is sent based on the user identifier described in the RTCP packet. The information of the user who is making a speech and the information of the user who is not making a speech are displayed in different formats.

  If it is determined in step S36 that no speech has been started, the processes in steps S37 to S40 are skipped.

  On the other hand, if it is determined in step S35 that there is a user who has finished speaking, the packet analysis unit 31 proceeds to step S41, and since it is determined in step S31 that there is a user who has left the conference, or in the conference. Since it is determined in step S32 that there is a user who has participated, it is determined whether or not it has been confirmed that the configuration of the user participating in the conference has changed.

  In step S41, when the packet analysis unit 31 determines that the configuration of the user participating in the conference has been confirmed, the process proceeds to step S42.

  In step S42, the packet generator 33 refers to the user list and rewrites the CSRC of the RTP header of the RTP packet to be transmitted and the CSRC of the RTCP packet.

  On the other hand, in step S41, the packet analysis unit 31 does not determine in step S31 that there is a user who has left the conference, and does not determine in step S32 that there is a user who has participated in the conference. If it is determined that it has not been confirmed that the configuration of the user participating in the process has changed, the process of step S42 is skipped.

  In step S43, the packet generator 33 deletes, from the SDES packet, the identifier of the user who is the main speaker who has been determined to have ended in step S35.

  Here, the SDES packet from which the identifier is deleted is transmitted to the user terminals 1-2 and 1-3 in step S40, and thereby the timing at which the configuration of the user who speaks at the same time is changed due to the termination of the certain user. Thus, the SDES packet is transmitted from the user terminal 1-1.

  When the RTP packet and the SDES packet are transmitted, the packet analyzing unit 31 determines whether or not there are any remaining users participating in the conference in step S44.

  If the packet analysis unit 31 determines in step S44 that there are users remaining in the conference, the packet analysis unit 31 returns to step S31 and repeatedly executes the above-described processing. On the other hand, since all the users have left, the packet analysis unit 31 remains in the conference. If it is determined that no user is present, the process is terminated.

  Next, processing of the user terminal 1-2 that controls display of user information will be described with reference to the flowchart of FIG. A process similar to the process shown in FIG. 12 is also performed in the user terminal 1-3.

  In step S51, the packet analysis unit 41 of the user terminal 1-2 determines whether the RTP packet and the SDES packet transmitted from the user terminal 1-1 have been received by the communication control unit 11, and determines that they have been received. If so, the process proceeds to step S52.

  In step S52, the packet analysis unit 41 acquires, for example, CSRC described in the RTP header of the RTP packet and CNAME described in the SDES packet as SDES ITEM from the data received by the communication control unit 11.

  In step S53, the packet analysis unit 41 determines whether or not the CSRC and CNAME acquired in step S52 have a change in data content compared to the stored CSRC and CNAME. In the packet analysis unit 41, CSRC described in the RTP packet transmitted from the user terminal 1-1 and CNAME described in the SDES packet as SDES ITEM are stored, and these data are, for example, RTP Packets and RTCP packets are newly received and rewritten whenever there is a change in content.

  When the configuration of the user participating in the conference changes, an RTP packet with the CSRC rewritten is transmitted, and at that time, it is determined that the content of the CSRC has changed. Further, when the configuration of a user who makes a statement at the same time is changed, an SDES packet in which a CNAME is added or deleted is transmitted. At this time, it is determined that the content of the CNAME has changed.

  If the packet analysis unit 41 determines in step S53 that the data contents have changed, the packet analysis unit 41 proceeds to step S54. At this time, for example, information indicating the user who is making a speech and the user who is not making a speech, which are specified from the CSRC of the RTP packet and the CNAME described in the SDES packet, is output to the input / output control unit 13.

  In step S54, based on the information supplied from the packet analysis unit 41, the input / output control unit 13 displays the information of the user who is speaking on the display 24 in a format different from the information of the user who is not speaking. Let

  FIG. 14 is a diagram illustrating an example of a screen displayed on the display 24 of the user terminal 1-2.

  In the example of FIG. 14, tabs 51 to 53 are displayed along the lower edge of the display 24. The tab 51 displays “Mr. A” representing the name of the user of the user terminal 1-1, and the tab 52 displays “Mr. B” representing the name of the user of the user terminal 1-2. The tab 53 displays “Mr. C” representing the name of the user of the user terminal 1-3.

  In the example of FIG. 14, among the tabs 51 to 53, the tabs 52 and 53 are displayed in a color different from that of the tab 51. The hatched lines attached to the tabs 52 and 53 indicate that the tabs 52 and 53 are displayed in a different color compared to the tabs 51 that are not hatched.

  That is, FIG. 14 illustrates an example in which a user B who is a user of the user terminal 1-2 and a user C who is a user of the user terminal 1-3 are speaking.

  From such a screen display, the user B can confirm that the sound output from the speaker 23 is the sound of the user and the user C. When the screen of FIG. 14 is displayed, the audio output unit 15 of the user terminal 1-2 outputs audio from the speaker 23 based on the audio data stored in the RTP packet acquired by the packet analysis unit 41. Has been.

  As shown in FIG. 14, information on all users who participate in the conference is displayed, and information on users who are speaking and information on users who are not speaking are not displayed in different formats. Alternatively, only the information of the user who is making a speech specified from the CNAME described in the SDES packet may be displayed.

  Returning to the description of FIG. 13, in step S55, the packet analysis unit 41 stores the CSRC and CNAME acquired in step S52, and proceeds to step S56. If it is determined in step S51 that an RTP packet and an SDES packet have not been received, or if it is determined in step S53 that there is no change in the contents of CSRC and CNAME, the process proceeds to step S56.

  In step S56, the packet analysis unit 41 determines whether to end participation in the conference.

  If the packet analysis unit 41 determines in step S56 that the participation in the conference is not terminated, the packet analysis unit 41 returns to step S51 and repeatedly executes the above-described processing. On the other hand, if the packet analysis unit 41 determines that the participation in the conference is terminated, the processing is performed. Terminate. For example, when the user B instructs to end the participation in the conference, it is determined that the participation in the conference is ended.

  Through the above processing, the user B who is a participant in the conference can easily confirm at a glance the user who is speaking and the user who is not speaking from the screen display.

  FIG. 15 is a block diagram illustrating another configuration example of the user terminal. The same components as those in FIG. 9 are denoted by the same reference numerals.

  In the example of FIG. 15, the same configuration as that of the user terminal 1-1 of FIG. 9 is realized by the two devices of the device 51 and the device 52. In the device 51, the communication control unit 11 and the main control unit 12 are realized, and in the device 52, the input / output control unit 13, the voice input unit 14, the voice output unit 15, and the display control unit 16 are realized.

  The main control unit 12 of the device 51 and the input / output control unit 13 of the device 52 are connected via a USB (Universal Serial Bus) cable or the like. The processing similar to the processing described above performed by the user terminal 1-1 having the configuration shown in FIG. 9 by the device 51 and the device 52 having such a configuration, or the processing described above performed by the user terminal 1-2. The same processing may be performed.

  In the example of FIG. 15, an LED (Light Emitting Diode) light emitting unit 53 is provided in the device 52 instead of the display 24 of FIG. 9. The LED light emitting unit 53 is provided with LEDs 53A to 53C arranged side by side. The light emission of the LED 53A indicates that the user A to whom the LED 53A is assigned speaks, and the light emission of the LED 53B indicates that the user B to which the LED 53B is assigned speaks. The light emission of the LED 53C indicates that the user of the user terminal 1-3 to which the LED 53C is assigned is speaking.

  In the example of FIG. 15, the LED 53B and the LED 53C emit light, which indicates that the user B and the user C are speaking at the same time.

  The user who is speaking and the user who is not speaking are not represented by the screen display, but can be represented by the light emission of the LED.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a program recording medium in a general-purpose personal computer or the like.

  FIG. 16 is a block diagram illustrating a configuration example of a personal computer that executes the above-described series of processing by a program.

  A CPU (Central Processing Unit) 101 executes various processes according to a program stored in a ROM (Read Only Memory) 102 or a storage unit 108. A RAM (Random Access Memory) 103 appropriately stores programs executed by the CPU 101 and data. These CPU 101, ROM 102, and RAM 103 are connected to each other by a bus 104.

  An input / output interface 105 is also connected to the CPU 101 via the bus 104. Connected to the input / output interface 105 are an input unit 106 made up of a keyboard, mouse, microphone, and the like, and an output unit 107 made up of a display, a speaker, and the like. The CPU 101 executes various processes in response to commands input from the input unit 106. Then, the CPU 101 outputs the processing result to the output unit 107.

  The storage unit 108 connected to the input / output interface 105 includes, for example, a hard disk, and stores programs executed by the CPU 101 and various data. A communication unit 109 communicates with an external device via the network 2.

  The drive 110 connected to the input / output interface 105 drives a removable medium 111 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and drives programs and data recorded there. Get etc. The acquired program and data are transferred to and stored in the storage unit 108 as necessary.

  As shown in FIG. 16, a program recording medium for storing a program that is installed in a computer and can be executed by the computer is a magnetic disk (including a flexible disk), an optical disk (CD-ROM (Compact Disc-Read Only). Memory, DVD (Digital Versatile Disc), a magneto-optical disk, a removable medium 111 that is a package medium composed of a semiconductor memory, or the ROM 102 in which a program is temporarily or permanently stored, or a storage unit 108 It is comprised by the hard disk etc. which comprise. The program is stored in the program recording medium using a wired or wireless communication medium such as a local area network, the Internet, or digital satellite broadcasting via a communication unit 109 that is an interface such as a router or a modem as necessary. Done.

  In the present specification, the steps for describing a program are not only processes performed in time series in the order described, but also processes that are executed in parallel or individually even if they are not necessarily processed in time series. Is also included.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

It is a figure which shows the structural example of the communication system which concerns on one Embodiment of this invention. It is a figure explaining the flow of transmission / reception of the data performed between the user terminals 1-1 thru | or 1-3 of FIG. It is a figure which shows the example of the data structure of an RTP packet. It is a figure which shows the example of the data structure of a RTCP packet. It is a figure which shows the example of the data described by the SDES packet. It is a figure which shows the example of SDES ITEM. It is a figure which shows Item Identifier and Item Description of FIG. It is a figure which shows the example of the identifier described by the SDES packet. It is a block diagram which shows the function structural example of the user terminal 1-1. It is a block diagram which shows the structural example of the main control part of FIG. It is a block diagram which shows the other structural example of the main control part of FIG. It is a flowchart explaining the conference management process of the user terminal 1-1. It is a flowchart explaining the display control process of the user terminal 1-2. It is a figure which shows the example of the screen displayed on a display. It is a block diagram which shows the other structural example of the user terminal 1-1. It is a block diagram which shows the structure of a personal computer.

Explanation of symbols

  1-1 to 1-3 User terminals, 2 networks, 3 servers, 11 communication control units, 12 main control units, 13 input / output control units, 14 audio input units, 15 audio output units, 16 display control units, 21 network terminals , 22 microphones, 23 speakers, 24 displays, 31 packet analysis units, 32 speech synthesis units, 33 packet generation units, 41 packet analysis units, 42 packet generation units

Claims (9)

In a communication system consisting of an information management device and a plurality of information processing devices connected via a network,
The information management device includes:
The voices obtained and transmitted by the predetermined information processing apparatuses are synthesized, and the voices obtained by synthesizing the voices are transmitted to the plurality of information processing apparatuses and output to the plurality of information processing apparatuses. A compositing means that allows conversations between
Detecting means for detecting the start or end of a user's speech based on the state of sound acquired by the plurality of information processing devices;
Each time the detection means detects the start or end of a user's speech, and the configuration of the user who speaks at the same time is changed, the first identification information for identifying the users of the plurality of information processing devices and the speech are performed. Generating means for generating control information including second identification information for identifying a user, and transmitting the control information to a plurality of the information processing devices;
The plurality of information processing apparatuses are respectively
Obtaining means for obtaining the control information transmitted from the information management device;
Each time the control information is acquired by the acquisition means, a display that identifies a user who is speaking and a user who is not speaking based on the control information, and displays information related to each user in different formats A communication system comprising control means. In an information management apparatus connected to a plurality of information processing apparatuses via a network,
The voices obtained and transmitted by the predetermined information processing apparatuses are synthesized, and the voices obtained by synthesizing the voices are transmitted to the plurality of information processing apparatuses and output to the plurality of information processing apparatuses. A compositing means that allows conversations between
Detecting means for detecting the start or end of a user's speech based on the state of sound acquired by the plurality of information processing devices;
Each time the detection means detects the start or end of a user's speech, and the configuration of the user who speaks at the same time is changed, the first identification information for identifying the users of the plurality of information processing devices and the speech are performed. An information management device comprising: generating means for generating control information including second identification information for identifying a user and transmitting the control information to a plurality of the information processing devices. The information management apparatus according to claim 2, wherein the control information is an RTCP packet, and the first identification information is a CSRC described in the RTCP packet. The information management apparatus according to claim 3, wherein the second identification information is information described in an SDES type RTCP packet as SDES ITEM. In an information processing method of an information management device connected to a plurality of information processing devices via a network,
The voices obtained and transmitted by the predetermined information processing apparatuses are synthesized, and the voices obtained by synthesizing the voices are transmitted to the plurality of information processing apparatuses and output to the plurality of information processing apparatuses. Have conversations between them,
Detecting the start or end of a user's speech based on the status of the sound acquired by the plurality of information processing devices;
Each time the start or end of a user's speech is detected and the configuration of the user who speaks at the same time is changed, the first identification information for identifying the users of the plurality of information processing devices and the user who is speaking are identified. An information processing method including a step of generating control information including second identification information and transmitting the control information to a plurality of the information processing devices. In a program for causing a computer to execute information processing of an information management device connected to a plurality of information processing devices via a network,
The voices obtained and transmitted by the predetermined information processing apparatuses are synthesized, and the voices obtained by synthesizing the voices are transmitted to the plurality of information processing apparatuses and output to the plurality of information processing apparatuses. Have conversations between them,
Detecting the start or end of a user's speech based on the status of the sound acquired by the plurality of information processing devices;
Each time the start or end of a user's speech is detected and the configuration of the user who speaks at the same time is changed, the first identification information for identifying the users of the plurality of information processing devices and the user who is speaking are identified. A program including a step of generating control information including second identification information and transmitting the control information to a plurality of the information processing apparatuses. Between the users of a plurality of information processing devices by synthesizing the voices acquired and transmitted by a predetermined information processing device and transmitting the synthesized voices to the plurality of information processing devices for output. A synthesizing unit that causes a conversation to be performed, a detecting unit that detects a start or end of a user's speech based on a voice situation acquired by a plurality of the information processing devices, and a start or end of the user's speech by the detecting unit Each time an end is detected and the configuration of a user who makes a statement changes at the same time, first identification information for identifying a plurality of users of the information processing apparatus and second identification information for identifying a user who makes a statement Information processing apparatus that includes control means for generating control information including and transmitting the control information to a plurality of information processing apparatuses together with other information processing apparatuses via an information processing apparatus In the location,
Obtaining means for obtaining the control information transmitted from the information management device;
Each time the control information is acquired by the acquisition means, a display that identifies a user who is speaking and a user who is not speaking based on the control information, and displays information related to each user in different formats An information processing apparatus comprising: control means. Between the users of a plurality of information processing devices by synthesizing the voices acquired and transmitted by a predetermined information processing device and transmitting the synthesized voices to the plurality of information processing devices for output. A synthesizing unit that causes a conversation to be performed, a detecting unit that detects a start or end of a user's speech based on a voice situation acquired by a plurality of the information processing devices, and a start or end of the user's speech by the detecting unit Each time an end is detected and the configuration of a user who makes a statement changes at the same time, first identification information for identifying a plurality of users of the information processing apparatus and second identification information for identifying a user who makes a statement Information processing apparatus that includes control means for generating control information including and transmitting the control information to a plurality of information processing apparatuses together with other information processing apparatuses via an information processing apparatus In the information processing method of the location,
Obtaining the control information transmitted from the information management device;
Information processing including a step of identifying a user who is making a speech and a user who is not making a speech based on the control information each time the control information is acquired, and displaying information related to each user in a different format Method. Between the users of a plurality of information processing devices by synthesizing the voices acquired and transmitted by a predetermined information processing device and transmitting the synthesized voices to the plurality of information processing devices for output. A synthesizing unit that causes a conversation to be performed, a detecting unit that detects a start or end of a user's speech based on a voice situation acquired by a plurality of the information processing devices, and a start or end of the user's speech by the detecting unit Each time an end is detected and the configuration of a user who makes a statement changes at the same time, first identification information for identifying a plurality of users of the information processing apparatus and second identification information for identifying a user who makes a statement Information processing apparatus that includes control means for generating control information including and transmitting the control information to a plurality of information processing apparatuses together with other information processing apparatuses via an information processing apparatus A program for executing information processing of location in the computer,
Obtaining the control information transmitted from the information management device;
A program including a step of identifying a user who is speaking and a user who is not speaking based on the control information each time the control information is acquired, and displaying information related to each user in a different format.

Images