JP5579660B2 - Multipoint video conferencing equipment - Google Patents

Description

  The present invention relates to a video conference apparatus, and more particularly to a multipoint connection video conference apparatus that autonomously determines a parent device.

  Currently, with the spread of the IP (Internet Protocol) network using DSL, optical fiber, etc., a technique for conducting a multipoint video conference using the IP network is known.

  As a method of realizing a multipoint connection video conference, audio data and video data transmitted from each device are received, mixing processing is performed on the audio data, video editing processing such as screen composition is performed on the video data, Patent Document 1 describes a technique using a dedicated device called MCU (Multi point Control Unit) that distributes processed audio data and video data to each device.

  On the other hand, with the recent improvement in processing capability of general-purpose processors, it has become possible to realize MCU functions and video conference device functions with a single general-purpose processor. As such a video conference apparatus, one described in Patent Document 2 is known.

Japanese Patent Application Laid-Open No. 07-066876 JP 2008-288974 A

  In the technique disclosed in Patent Document 2, a terminal serving as a parent device is determined in advance, and the child device must send a call to the parent device, and the child device not previously given the parent device information participates in the conference Can not do it.

  The present invention solves the above-described problems, and an object thereof is to make it possible to flexibly implement a multipoint connection video conference.

  In order to solve the above-described problem, the multipoint connection video conference apparatus uses SDP (Session Description Protocol) and RTP (Real Time Protocol) so that each apparatus notifies the connection destination apparatus of its own connection state. Determine the parent machine. ADVANTAGE OF THE INVENTION According to this invention, even if the information of a main | base station and a subunit | mobile_unit is not given previously, a multipoint connection video conference can be implemented flexibly.

  A multipoint videoconferencing device has a slave unit that is newly connected to a multipoint videoconferencing device connected to a device that is already operating as a slave unit, and the master unit competes in the multipoint videoconferencing system. If a wrong connection is made, the handset that received the call based on the SIP-URI (Session Initiation Protocol-Uniform Resources Identifiers) included in the SIP message forwards the call to the base unit. By instructing, a multipoint connection video conference can be performed with a large number of devices without the user being aware of the master unit and the slave unit.

  The above-described problems are a line IF that realizes input / output of data from the IP network, an audio input IF for capturing audio data from a microphone, an audio compression unit for compressing data from the audio input IF, An audio decompression unit for decompressing compressed audio data, an audio synthesis unit for synthesizing audio transmitted from multiple points, an audio output IF unit for outputting audio data to a speaker, and a camera A video input IF for capturing video data, a video decoding unit for decoding data from the video input IF, a video scaling unit for scaling the decoded video data to a predetermined size, and for compressing the video data Video compression unit, a video expansion unit for expanding compressed video data, and a video unit for combining video data transmitted from multiple points. Compositing unit, video scaling unit for scaling the synthesized video to a fixed size, video encoding unit for encoding the scaled video, and video output for outputting the encoded video data to the TV In a multipoint videoconferencing device comprising a RAM that stores IF and SDP, RTP, and SIP information, other multipoint videoconferencing devices participate in the videoconference during a one-on-one videoconference. This can be achieved by a multipoint connection video conference apparatus having a communication control unit that dynamically determines the master unit in the multipoint connection videoconference system using the SDP and RTP received by the line IF.

  Also, when a join message is received, when the status is not connected, a response message is written indicating that the status is undetermined, and when the status is undetermined or parent status, the response message is described as being the parent status. This can be achieved with a multipoint videoconferencing device that responds with

  In addition, when a join message is received with the sender status being undetermined, when the status is not connected, the response message describes that the status is undetermined and returns to the sender, and the status transitions to the undetermined status. When the status is undetermined, the response message describes the parent status and returns to the sender. When the status is the parent status, the response message indicates that the status is the parent status. When the status is in the child state, the redirect message is sent back to the sender, and when the join status message is received in the parent status, the response message is in the child status when the status is not connected. Achieved by multipoint video conferencing equipment that writes and returns to the sender, transitions to the child state, and returns a busy message to the sender when the state is undecided, parent, or child Can .

  The multipoint connection video conference apparatus can dynamically determine the parent device in the system using each protocol of SDP, RTP, and SIP. In addition, even if the device makes a connection in which the parent device competes in the system, the newly connected child device is transferred to the parent device, and the multipoint connection video conference is performed flexibly and reliably. .

It is a block diagram explaining the structure of a multipoint connection video conference system. It is a block diagram explaining the structure of a multipoint connection video conference apparatus. This is an SDP description of the INVITE message body. It is a figure explaining a control code identifier and notification information. It is a sequence diagram between two multipoint connection video conference apparatuses. It is a figure explaining RTP for control. It is a figure explaining the defined setting content. It is a sequence diagram between three multipoint connection video conference apparatuses. It is a flowchart of the incoming operation | movement of a multipoint connection video conference apparatus (the 1). It is a flowchart of the incoming operation | movement of a multipoint connection video conference apparatus (the 2). It is a flowchart of the incoming operation | movement of a multipoint connection video conference apparatus (the 3). It is a flowchart of the transmission operation | movement of a multipoint connection video conference apparatus. It is a transmission SDP information condition table by the own apparatus connection state at the time of transmission. It is a sequence diagram between four multipoint connection video conference apparatuses (the 1). It is a figure explaining the apparatus information which a multipoint connection video conference apparatus manages. It is a sequence diagram between four multipoint connection video conference apparatuses (the 2). It is a figure explaining the case where there was a connection from another multipoint connection video conference during implementation of a multipoint connection video conference. It is the operation condition table by the state of an apparatus and the received SDP information. It is a sequence diagram between two multipoint connection video conference apparatuses in a parent state. It is a figure explaining the case where a main machine leaves a seat during multipoint connection video conference execution. It is a sequence diagram between four multipoint connection video conference apparatuses (the 3). It is an operation | movement flowchart when the multipoint connection video conference apparatus in a parent state leaves.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings using examples. Note that the same reference numerals are assigned to substantially the same parts, and description thereof will not be repeated.

  The multipoint connection video conference apparatus according to the present embodiment has both an MCU function (master unit) and a video conference apparatus function (slave unit) when a multipoint connection video conference is performed. However, the multipoint connection video conference apparatus performs the multipoint connection video conference with a large number of apparatuses without being aware of the master unit and the slave unit. Note that in this specification or the drawings, a multipoint video conference device may be simply referred to as a device or a multipoint video conference device.

  When a multipoint videoconferencing device operates as a parent device, the input video data from the camera connected to the device and the video data received from another device are combined and scaled, and the video data is sent to the child device. Distribute and output to monitor connected to own device. Similarly, the voice data from the microphone connected to the own apparatus and the voice data received from the other apparatus are mixed, the voice data is distributed to the slave unit, and output to the speaker connected to the own apparatus.

  When a multipoint videoconferencing device operates as a slave unit, input video data from the camera connected to the host device is transmitted to the master unit, and the composite video data distributed from the master unit is connected to the host device. Output to the monitor. Also, the input voice data from the microphone connected to the own device is transmitted to the parent device, and the voice data distributed from the parent device is output to the speaker connected to the own device.

  The multipoint connection video conference apparatus performs call control between apparatuses connected by an IP network using SIP standardized by IETF (The Internet Engineering Task Force), thereby realizing a multipoint connection video conference.

  With reference to FIG. 1, a network configuration of a multipoint connection video conference apparatus system will be described. In FIG. 1, a multipoint connection video conference apparatus system 100 includes a multipoint connection video conference apparatus 10 and an IP network 20. The multipoint connection video conference apparatus 10 is connected to an IP network 20 that transmits packets of video data, audio data, and the like. The multipoint connection video conference apparatus 10 is connected to a camera 11, a monitor 12, a microphone 13, and a speaker 14. The camera 11 distributes the video of its own site. The monitor 12 displays images from multiple points. The microphone 13 distributes the voice of the local site. The speaker 14 outputs multipoint sound.

  With reference to FIG. 2, an internal configuration of the multipoint connection video conference apparatus will be described. In FIG. 2, the multipoint connection video conference apparatus 10 includes a line IF 101, a communication control unit 102, a RAM 103, a voice input IF 104, a voice compression unit 105, a voice expansion unit 106, a voice synthesis unit 107, and a voice. The output IF 108, the video input IF 109, the video decoding unit 110, the video scaling unit 111, the video compression unit 112, the video expansion unit 113, the video synthesis unit 114, the video scaling unit 115, and the video encoding unit 116 , And a video output IF 117.

  The line IF 101 is an interface that performs data communication using the IP network 20 between the multipoint connection video conference apparatus and another multipoint connection video conference apparatus. The line IF 101 outputs the data received from the IP network 20 to the communication control unit 102. Further, the line IF 101 outputs the data received from the communication control unit 102 to the IP network 20.

  The communication control unit 102 includes a processor and the like. The communication control unit 102 performs SIP control used for communication between multipoint connection television apparatuses, SDP control for notifying the state of the apparatus, and RTP control including video and audio data and control data. The communication control unit 102 inputs and outputs these data to the RAM 103. In addition, the communication control unit 102 outputs the compressed audio data to the audio expansion unit 106 and outputs the compressed video data to the video expansion unit 113.

  The RAM 103 is a memory for storing SIP-URI information included in SDP, RTP, and SIP messages transmitted from the connection destination multipoint connection video conference apparatus via the IP network 20. Based on the SDP, RTP, and SIP-URI data stored in the RAM 103, the multipoint connection video conference apparatus 10 determines a parent-child relationship between the multipoint connection video conference apparatuses.

  The audio input IF 104 is an interface for capturing audio data input from the microphone 13 connected to the apparatus. The voice compression unit 105 compresses the voice data input from the voice input IF 104 so as not to place a load on the IP network 20 and outputs the compressed data to the communication control unit 102. The audio expansion unit 106 expands the audio data compressed by the audio compression unit 105. When the device functions as a parent device, the voice decompression unit 106 outputs the decompressed voice data to the voice synthesis unit 107. On the other hand, when functioning as a slave unit, the audio decompression unit 106 outputs the decompressed audio data to the audio output IF 108.

  The speech synthesizer 107 functions only when the device is a parent device. The voice synthesizer 107 synthesizes voice data from each slave unit and from the microphone 13 of the own apparatus, and outputs the synthesized voice data to the voice output IF 108 and the communication controller 102. The audio output IF 108 outputs the audio data processed in the apparatus to the speaker 14 connected to the apparatus.

  The video input IF 109 is an interface for capturing video data input from a camera connected to the apparatus. The video decoding unit 110 decodes the video data input from the video input IF 109 and outputs the decoded video data to the video scaling unit 111. The video scaling unit 111 scales the video data output from the video decoding unit 110 according to the connection form, and outputs it to the video compression unit 112.

  The video compression unit 112 compresses the data output from the video scaling unit 111 so as not to place a load on the IP network 20 and outputs the compressed data to the communication control unit 102. The video decompression unit 113 decompresses the video data compressed by the video compression unit 112. When the apparatus functions as a master unit, the video decompression unit 113 outputs the decompressed video data to the video composition unit 114. When functioning as a slave, the video decompression unit 113 outputs the decompressed video data to the video scaling unit 115.

The video composition unit 114 functions only when the device serves as a parent device. The video synthesizing unit 114 synthesizes video data from each slave unit and the camera 11 of its own device and outputs the synthesized video data to the video scaling unit 115. The video scaling unit 115 scales the video data output from the video synthesis unit 114 according to the connection form, and outputs the scaled video data to the video encoding unit 116 and the communication control unit 102. Note that the video scaling unit 115 also outputs to the communication control unit 102 in the case of the parent device.

  The video encoding unit 116 outputs the video data from the video scaling unit 115 to the video output IF 117. The video output IF 117 outputs video data processed in the apparatus to the monitor 12 connected to the apparatus.

  SIP, SDP, and RTP including control information for realizing a multipoint connection video conference will be described. With reference to FIG. 3, the SDP description described in the body of the INVITE message transmitted by the multipoint connection video conference apparatus will be described. In FIG. 3, from the v line of the SDP description to a = framerate: 15 are standard descriptions. The suffix a = conference: server is the control code and notification information newly defined in this embodiment. Specifically, conference is a control code identifier, and server is notification information.

  The multipoint connection video conference apparatus 10 performs communication negotiation by SDP at the timing of outgoing / incoming and response. An attribute description a for transmitting the state of the own device is added to the SDP media description as the contents to be notified, the state of the own device is set in the attribute description a, and is transmitted to the connected multipoint connection video conference device 10.

  With reference to FIG. 4, the setting correspondence table 30 in which the control code identifier and the notification information are described will be described. In FIG. 4, the setting correspondence table 30 includes a control code identifier 301 record and a notification information 302 record. The setting correspondence table 30 includes a setting content column and a meaning column corresponding to the record. The setting content of the control code identifier 301 is one and only the conference. The conference indicates negotiation data. On the other hand, there are three types of setting contents of the notification information 302. They are undecided, server, and client.

  Undecided means that the state of the device itself is “undecided”. Undecided means a one-to-one connection. “server” means that the state of the own device is “parent” of one-to-multiple connection. The client means that the state of the own device is a “child” of one-to-multiple connection.

  Note that the setting values of the control code identifier 301 and the notification information 302 can be arbitrarily determined. Another value may be set as the set value. As shown in FIG. 4, a fixed value called “conference” is used as the control code identifier 301, and negotiation with the SDP is attempted only with the device using this control code identifier 301.

  With reference to FIG. 5, a connection process between two multipoint connection video conference apparatuses will be described. This connection process is an outgoing / incoming sequence using SDP. In FIG. 5, the device 10-A not communicating with another device similarly makes a call connection request to the device 10-B not communicating with another device. At that time, according to the setting correspondence table 30 of FIG. 4, the device 10-A and the device 10-B are connected to each other from a non-communication and have a one-to-one connection. The transmitted INVITE message is transmitted to the apparatus 10-B (S1). Since the device 10-B also has a one-to-one connection, according to the setting correspondence table 30 in FIG. 4, the communication control unit 102 of the device 10-B sends a normal response 200 OK with the attribute information a in SDP undecided to the device 10-A. (S2). At this time, the communication is one-to-one, and there is no parent-child relationship between the devices.

  As described with reference to FIG. 5, the communication state of the apparatus can be notified by using SDP for transmission and its response, and communication negotiation can be performed. However, if the state of the device changes during communication, SDP cannot be used for negotiation with the other party's video conference device. This is because the SIP message including the SDP is a transmission of only the INVITE message and the 200 OK message at the start of the session. Therefore, apart from the voice RTP, a message notifying the state of the apparatus is interrupted in the RTP as the control RTP shown in FIG. This RTP negotiation is performed every time the state of the device changes after connection.

  The control RTP data format and the setting correspondence table 40 will be described with reference to FIGS. In FIG. 6, the conference before the first colon of the control: conference: request: server is a control code identifier. The request between the colons is an instruction code. The server after the second colon is notification information.

  In FIG. 7, the setting values of the control code identifier 401, the instruction code 402, and the notification information 403 can be arbitrarily determined. The setting values in FIG. 7 may be set to other values. The control RTP includes a control code identifier 401, an instruction code 402, and notification information 403. The control code identifier 401 uses a fixed value called “conference”, and negotiates with RTP only with the device using this control code identifier 401.

  There are two types of instruction code 402: a request that is transmitted to notify the state from a device that transmits control RTP, and a response that is transmitted by a device that receives control RTP to indicate a response to the request.

  There are four types of notification information 403, undecided indicating that the parent-child relationship between devices in a one-to-one connection state is undecided, server indicating a parent device when performing multipoint connection, and multipoint There is a client indicating that it is a slave when connecting, and xxx that is transmitted only as a response when a request cannot be accepted.

  With reference to FIG. 8, the parent-child relationship negotiation between apparatuses using the control RTP described in FIGS. 6 and 7 will be described. In FIG. 8, one-to-one communication between the device 10-A and the device 10-B is performed in the initial state. The device states of the devices 10-A and 10-B are both undecided. Here, the device 10-C transmits an INVITE message in which the SDP internal attribute information a is undecided to the device 10-A (S3). The device 10-A transmits a normal response 200 OK message to the INVITE message from the device 10-C to the device 10-C (S4). The attribute description a of the SDP transmitted in step 4 is “server”, and the device status of the device 10 -A changes from undecided to server when the device 10 -C connects to the device 10 -A.

  As described above, the apparatus 10-A transmits the notification information server to the apparatus 10-B by the control RTP in response to the change in the state of the apparatus 10-A that becomes the parent device during communication (S5). The device 10-B transmits the notification information client as a response using the control RTP (S6). As a result, the parent-child relationship among the devices 10-A, 10-B, and 10-C is determined, and transition is made during communication.

  With reference to FIG. 9, how to determine the parent-child relationship between the video conference apparatuses will be described. The process of the flowchart in FIG. 9 describes the operation of the communication control unit 102 using the SDP attribute information data from the communication destination device stored in the RAM 103 of the device 10.

  In FIG. 9, the communication control unit 102 receives an incoming call from another video conference device (S100). The communication control unit 102 determines whether or not the own device is communicating with another device (S101). When not in communication (NO), the communication control unit 102 determines whether or not the received SDP attribute information is undecided (S102). If it is determined as undecided (YES), the communication control unit 102 notifies the other terminal as undecided as SDP attribute information (S104), and the process ends. At this time, the communication is one-to-one, and the parent-child relationship between the devices is undecided.

  On the other hand, if it is determined in step 102 that the received SDP attribute information is not undecided (NO), the communication control unit 102 determines whether or not the received SDP attribute information is a server (S103). . If it is determined as “server” (YES), the communication control unit 102 notifies the partner terminal of “client” as SDP attribute information (S105), and the process ends. Here, since there is a one-to-multiple connection, the parent-child relationship between devices is determined.

  On the other hand, if it is determined in step 103 that the received SDP attribute information is not a server (NO), the received SDP attribute information is client. However, in the present embodiment, because the contention of the parent device occurs, the child device (client) is not allowed to transmit, and the SDP in which the attribute information is declared client is not received. . For this reason, the connection becomes impossible and the communication control unit 102 ends the flow. If the communication control unit 102 determines in step 101 that the own device is communicating with another device (YES), the communication control unit 102 transitions to step 106.

  9B, in step 106, the communication control unit 102. It is determined whether or not the connection state of the own device is undecided. When it is determined that the connection state of the own device is undecided (YES), the communication control unit 102 determines whether the received SDP attribute information is undecided (S107). When it is determined as undecided (YES), the communication control unit 102 notifies the partner apparatus of “server” as SDP attribute information (S109). At this time, one-to-multiple communication is performed, and the parent-child relationship between devices is determined. The communication control unit 102 notifies the server of another server that has transmitted undecided as SDP attribute information at the start of communication by using the control RTP (S110), and ends.

  On the other hand, when it is determined in step 107 that the received SDP attribute information is not undecided (NO), the communication control unit 102 determines whether the received SDP attribute information is a server (S108). If it is determined as server (YES), the communication control unit 102 transmits 486 busy, which is a SIP busy message, to the device that has requested connection in order to prevent contention of the parent device (S111), and ends. To do.

  On the other hand, if it is determined in step 108 that the server is not a server (NO), the received SDP attribute information is client. However, when a slave device (client) transmits, it is not allowed because the contention of the master device occurs, and an SDP in which attribute information is declared client is never received. For this reason, the connection becomes impossible and the communication control unit 102 ends the flow.

  If it is determined in step 106 that the connection state of the own device is not undecided (NO), the communication control unit 102 determines whether the connection state of the own device is server (S112). When it is determined that the server is a server (YES), the communication control unit 102 determines whether the received SDP attribute information is undecided (S113). If it is determined as undecided (YES), the communication control unit 102 notifies the partner device of “server” as SDP attribute information (S115), and the process ends. At this time, one-to-multiple communication has already been established, and the parent-child relationship of the devices has been determined.

  On the other hand, if it is determined in step 113 that it is not undecided (NO), the communication control unit 102 determines whether the received SDP attribute information is a server (S114). If it is determined as “server” (YES), the communication control unit 102 transmits a 486 busy message, which is a SIP busy message, to the device that has requested connection in order to prevent contention of the parent device (S116), and ends. To do.

On the other hand, if it is determined in step 114 that the server is not a server (NO), the received SDP attribute information is client. However, because the contention of the parent device occurs that the child device (client) transmits, the SDP in which the attribute information is declared client is not received because it is not allowed. For this reason, the connection becomes impossible and the communication control unit 102 ends the flow.
If the communication control unit 102 determines in step 112 that the connection state of the own apparatus is not “server” (NO), the process proceeds to step 118.

  In FIG. 9C, the communication control unit 102 determines whether the received SDP attribute information is undecided (S118). If it is determined as undecided (YES), the communication control unit 102 transmits a 302 Moved Temporary message, which is a SIP redirect message, to the partner device to connect to the parent device (S120), and the process ends.

  On the other hand, if it is determined in step 118 that it is not undecided (NO), the communication control unit 102 determines whether the received SDP attribute information is server (S119). If it is determined as server (YES), the communication control unit 102 transmits 486 busy, which is a SIP busy message, to the device that has requested connection in order to prevent contention of the parent device (S121), and ends. To do.

  On the other hand, if it is determined in step 119 that the server is not a server (NO), the received SDP attribute information is client. However, it is not allowed for the slave unit (client) to transmit because the conflict of the parent unit occurs, and the SDP in which the attribute information is declared as client is not received. For this reason, the communication control part 102 complete | finishes a flow.

  With reference to the flowchart of FIG. 10, how the parent-child relationship between the video conference apparatuses is determined when the apparatus transmits is described. The process of the flowchart in FIG. 10 describes the operation of the communication control unit 102 using the SDP attribute information data from the communication destination device stored in the RAM 103 of the device.

  In FIG. 10, the communication control unit 102 determines whether or not the own device is communicating with another device (S130). If it is determined that communication is not in progress (NO), the communication control unit 102 transmits an INVITE message with the attribute description in SDP as undecided to the connection destination device (S131), and the process ends.

  If it is determined in step 130 that communication is in progress (YES), the communication control unit 102 determines whether the communication state of the own device is undecided (S132). If it is determined as undecided (YES), the communication control unit 102 changes the communication state to “server” (S135). The communication control unit 102 transmits an INVITE message with the SDP attribute description as server to the connection destination device (S136), and the process ends. Note that the order of step 135 and step 136 may be reversed.

  If it is determined in step 132 that it is not undecided (NO), the communication control unit 102 determines whether the state of the own device is “sever” (S133). If it is determined as “server” (YES), the communication control unit 102 transmits an INVITE message in which the SDP internal attribute description is “server” to the connection destination device (S134), and ends.

  In step 133, when it is not the server, the communication control unit 102 ends as it is. This is because when it is neither undecided nor server, it becomes a client and transmission of the client is not permitted.

  With reference to FIG. 11, the transmission SDP information condition table 50 according to the connection state of the own device at the time of transmission will be described. In FIG. 11, the connection state includes uncommunication and communication. The communication status further includes undecided, server, and client. The SDP information when transmitting INVITE from these four states is as follows.

  When there is no communication, the communication control unit 102 transmits undecided. When undecided, the communication control unit 102 transmits a server. In the case of server, the communication control unit 102 transmits server. In the case of client, the communication control unit 102 does not transmit and therefore does not occur.

  With reference to FIG. 12, the connection sequence at the time of apparatus 10-A-10-D having a video conference in four places is demonstrated. In FIG. 12, the device 10-B sends a SIP INVITE message, which is a call connection request, to the device 10-A (S7). At this time, the attribute description is transmitted as undecided in the SDP of the body of the NVITE message. Upon receiving the INVITE message, the device 10-A transmits a normal response 200 OK message corresponding to the INVITE message to the device 10-B (S8). At this time, the attribute description is transmitted as undecided in the SDP of the body of the message.

  Upon receiving the normal response 200 OK message, device 10-B transmits an acknowledgment ACK message corresponding to the normal response 200 OK message to device 10-A (S9).

  The device 10 -A stores the attribute description in the SDP received from the device 10 -B in the RAM 103. Similarly, the device 10-B also saves the attribute description in the SDP in the RAM 103.

  After the one-to-one connection is completed between the devices 10-A and 10-B, the device 10-A transmits the audio data of the microphone 13 connected to the device 10-A and the video data of the camera 11 to the device 10-B. Send to. Similarly, the device 10-B transmits the audio data of the microphone 13 connected to the device 10-B and the video data of the camera 11 to the device 10-A. The device 10-A outputs the received data from the device 10-B to the speaker 14 and the monitor 12 connected to the device 10-A. The data from the device 10-A received by the device 10-B is output by the device 10-B to the speaker 14 and the monitor 12 connected to the device 10-B.

  After the one-to-one connection is completed between the devices 10-A and 10-B, the device 10-C sends a SIP INVITE message as a call connection request to the device 10-A (S10). The SDP attribute description of the body of the INVITE message is undecided.

  Upon receiving the INVITE message, the device 10-A transmits a normal response 200 OK message corresponding to the INVITE message to the device 10-C (S11). At this time, since the one-to-two connection is established and the device 10-A becomes a master unit that synthesizes and distributes audio and video, the attribute description declared in the SDP transmitted by the device 10-A is server.

  When receiving the normal response 200 OK message, the device 10-C transmits an acknowledgment ACK message corresponding to the normal response 200 OK message to the device 10-A (S12).

  At this time, since the one-to-two connection is made with the device 10-A as the parent device, the parent-child relationship is determined for the first time between the devices. On the basis of the one-to-two connection of the multipoint connection video conference system, based on the determined parent-child relationship, the device 10-A serving as the parent device has the attribute description a on the RAM 103 as undecided. A control RTP is used for B, and a server is transmitted as a request notifying that the device 10-A has become a parent device (S13). The device 10-B transmits client as a response to the request to the device 10-A (S14), and the connection state information of the RAM 103 of the device 10-B is updated.

  After the one-to-two connection is completed in the device 10-A, the device 10-B, and the device 10-C, the device 10-B transmits the audio data of the microphone 13 connected to the device 10-B and the video data of the camera 1. Similarly, the apparatus 10-C transmits the audio data of the microphone 13 connected to the apparatus 10-C and the video data of the camera 11 to the apparatus 10-A.

  The apparatus 10-A includes the audio data of the microphone 13 connected to the apparatus 10-A, the video data of the camera 11, the audio data of the microphone 13 received from the apparatus 10-B, the video data of the camera 11, and the apparatus 10- The audio data of the microphone 13 and the video data of the camera 11 received from C are respectively synthesized.

  Audio data and video data synthesized by the device 10-A are output by the speaker 14 and the monitor 12 connected to the device 10-A, respectively. The synthesized audio data and video data are also transmitted to the devices 10-B and 10-C at the same time. The device 10-B outputs the audio data and video data received from the device 10-A to the speaker 14 and the monitor 12 to which the device 10-B is connected. The device 10 -C outputs the audio data and video data received from the device 10 -A to the connected speaker 14 and monitor 12.

  After the one-to-two connection with the device 10-A as a parent is completed, the device 10-D sends a SIP INVITE message as a call connection request to the device 10-A (S15). The SDP attribute description of the body of the INVITE message is undecided.

  Upon receiving the INVITE message, the device 10-A transmits a normal response 200 OK message corresponding to the INVITE message to the device D (S16). The attribute description in the SDP of the message body is server. Upon receiving the normal response 200 OK message, the device 10-D transmits an acknowledgment ACK message corresponding to the normal response 200 OK message to the device 10-A (S17).

  After the one-to-three connection with the device 10-A as a parent is completed, the device 10-B sends the audio data of the microphone 13 connected to the device 10-B and the video data of the camera 11 to the device 10-A. Send. Similarly, the device 10-C transmits the audio data of the microphone 13 connected to the device 10-C and the video data of the camera 11 to the device 10-A. Similarly, the device 10-D transmits the audio data of the microphone 31 connected to the device 10-D and the video data of the camera 11 to the device 10-A.

  The apparatus 10-A includes the audio data of the microphone 13 connected to the apparatus 10-A, the video data of the camera 11, the audio data of the microphone 13 received from the apparatus 10-B, the video data of the camera 11, and the apparatus 10- The audio data of the microphone 13 received from C, the video data of the camera 11, the audio data of the microphone 13 received from the apparatus 10 -D, and the video data of the camera 11 are synthesized.

  Audio data and video data synthesized by the device 10-A are output by the speaker 14 and the monitor 12 connected to the device 10-A, respectively. The synthesized audio data and video data are also transmitted to the devices 10-B, 10-C, and 10-D at the same time. The device 10-B outputs the audio data and video data received from the device 10-A to the speaker 14 and the monitor 12 to which the device 10-B is connected. The device 10 -C outputs the audio data and video data received from the device 10 -A to the connected speaker 14 and monitor 12. The device 10-D outputs the audio data and video data received from the device 10-A to the connected speaker 14 and monitor 12.

  With reference to FIG. 13, a table showing information of each device managed on the RAM 103 by the multipoint connection video conference device will be described. When the sequence in FIG. 12 is completed up to step 17 and the one-to-three connection with the device A as a parent is completed, each device manages data as shown in FIG. That is, the device A manages the SIP-URIs of the devices B, C, and D and that the devices B, C, and D are clients. The devices B, C, and D manage the SIP-URI of the device A and that the device A is a server, respectively.

  With reference to FIG. 14, processing when a new device 10 -D transmits to the device 10 -C functioning as a slave during a one-to-two multipoint connection video conference will be described. In FIG. 14, the device 10-A functions as a master unit. Steps 18 to 25 are the same as steps 7 to 14 in FIG.

  The device 10-D sends a SIP INVITE message, which is a call connection request, to the device 10-C (S26). The SDP attribute description of the message body is undecided. The device 10-C receives the SDP whose attribute description is undecided. However, if the device is operating as a client and the incoming call is permitted, contention for the parent device occurs in the multipoint television system. Therefore, the communication control unit 102 of the device 10-C changes the connection of the device 10-D to the device 10-A that is currently the parent device in the multipoint connection video conference system with respect to the device 10-D. Therefore, 302 Moved Temporary which is a SIP redirect message is transmitted to the apparatus 10-D (S27). Note that the connection destination change information is a SIP-URI in a table indicating each piece of device information managed on the RAM 103 of the device 10-C, and the SIP-URI corresponding to the device 10-A is 302 Moved Temporary message. Specified in the contact header.

Upon receiving the redirect message 302 Moved Temporary, the device 10-D transmits an acknowledgment message ACK message corresponding to the redirect message 302 Moved Temporary to the device 10-C. (S28)
The device 10-D refers to the SIP-URI in the contact header in the 302 Moved Temporary message received from the device 10-C, and transmits the INVITE message again to the device 10-A that is the transfer destination (S29). The SDP attribute description of the message body is undecided.

  Upon receiving the INVITE message, the device 10-A transmits a normal response 200 OK message corresponding to the INVITE message to the device 10-D (S30). The SDP attribute description of the message body is “server”. Upon receiving the normal response 200 OK message, the device 10-D transmits an acknowledgment ACK message corresponding to the normal response 200 OK message to the device 10-A (S31).

  By the above method, the device 10-A is determined as the parent device, and the devices 10-B, 10-C, and 10-D are determined as the child devices, and the one-to-four connection is realized in the multipoint connection video conference. In this embodiment, the case of one-to-four connection has been described. However, even if the number of connections changes, the method for determining the parent device does not change.

  Hereinafter, the network in which the devices 10-C, 10-D, and 10-E perform one-to-two communication joins the network in which the devices 10-A and 10-B perform one-to-one communication. The processing in the case of having been performed will be described.

Referring to FIG. 15, devices 10-C, 10-D, and 10-E perform one-to-two communication in a network in which devices 10-A and 10-B perform one-to-one communication. Explain the case where a network is participating. In FIG. 15, since the device 10-A and the device 10-B perform one-to-one communication, the device 10-A and the device 10-B are in an undecided state. On the other hand, the device 10-C, the device 10-D, and the device 10-E communicate one-to-two, the device 10-C is in the server state, and the device 10-D and the device 10-E are in the client state. . Further, the device 10-C is transmitting to the device 10-A. As described above, in this case, the device 10-A that receives an incoming call becomes the parent device. However, since the transmitting device 10-C is also a parent device, there are two parent devices in the multipoint connection video conference system, and the parent devices compete with each other.

  With reference to FIG. 16, the operation condition table 60 based on the received attribute description in the SDP will be described in order to prevent contention of the parent device. In FIG. 16, the state of the video conference apparatus 10 that has received INVITE is not communicating or communicating. The communication state further includes undecided, server, and client. The SDP information when INVITE is received from these four states is as follows.

First, since the client does not allow the SDP information when INVITE is received, it does not occur.
When the SDP information when receiving INVITE is undecided and the state is not connected, the multipoint connection video conference apparatus 10 returns an undecided response and transitions to the undecided state. If the state is undecided, the multipoint connection video conference apparatus 10 returns a server response and transitions to the server state. If the state is server, the multipoint connection video conference apparatus 10 returns a server response. If the state is client, the multipoint video conference apparatus 10 transmits 302 Moved Temporary.

  When the SDP information when receiving the INVITE is “server” and the state is not connected, the multipoint connection video conference apparatus 10 returns a client response and transitions to the client state. If the state is undecided, server, or client, the multipoint connection video conference apparatus 10 transmits 486 busy.

With reference to FIG. 17, the connection sequence of the multipoint connection video conference apparatus 10-A and the multipoint connection video conference apparatus 10-C when the connection of FIG. 15 occurs will be described. In FIG. 17, the device 10-A receives the connection request INVETE message from the device 10-C (S36). However, since the attribute description in the SDP transmitted by the device 10-C is “server” and the device state of the device 10-A itself is undecided, the device 10-A is busy with the device 10-C. A 486 busy message indicating this is transmitted, and the connection is rejected (S37). The device 10-C that has received the 486 busy message transmits an acknowledgment message ACK message to the device 10-A (S38).
Also for the multipoint connection of FIGS. 12 and 14, the device 10-A, the device 10-B, the device 10-C, and the device 10-D operate according to the table of FIG.
In this way, it is possible to connect to any device flexibly without any complicated work of preventing the conflict of the parent device on the multipoint connection video conference system and determining the parent device between the conference participants in advance before the start of the conference. it can.

  Referring to FIGS. 18 to 20, device 10-A is used as a parent device, and device 10-A⇔device 10-B, device 10-A⇔device 10-C, device 10-A⇔device 10-D A process when the parent device 10-A leaves the conference during the 3 to 3 multi-point connection video conference will be described.

  Referring to FIG. 18, the device 10-A is the parent device during the one-to-three multipoint connection video conference of device A⇔device B, device A⇔device C, and device A⇔device D using the device 10-A as the parent device. A case where the apparatus 10-A leaves the conference will be described. In FIG. 18, when the multipoint connection video conference apparatus 10 -A, which is the parent apparatus, leaves, the multipoint connection video conference apparatus 10 -B, the multipoint connection video conference apparatus 10 -C, and the multipoint connection video conference apparatus 10 -Disconnect from D. The multipoint connection video conference device 10-C transitions from the child device to the parent device.

  With reference to FIG. 19, processing when the multipoint connection video conference is continued between the devices 10 -B, 10 -C, and 10 -D even after the device 10 -A leaves the seat will be described. In FIG. 19, the device 10 -A transmits BYE to the next device 10 -C as the parent device (S 41). The device 10-C transmits 200 OK to the device 10-A (S42). The device 10-A transmits REFER Refer-To: device C to the device 10-B (S43). The device 10-B transmits 202 Accepted to the device 10-A (S44). The device 10-A transmits BYE to the device 10-B (S46). The device 10-B transmits 200 OK to the device 10-A (S47).

  The device 10-A transmits a control RTP to the device 10-D (S48). At this time, the control data is conference: request: undecided. The device 10-D transmits a control RTP to the device 10-A (S49). At this time, the control data is conference: response: undecided.

  The device 10-B transmits INVITE Referred-By: device A to the device 10-C (S51). At this time, undecided is described in the SDP of the message body. The device 10-C transmits 200 OK to the device 10-B (S52). At this time, undecided is described in the message body. The device 10-B transmits ACK to the device 10-C (S53). The device 10-B transmits NOTIFY to the device 10-C (S54). NOTIFY is a message notifying the transfer. The device 10-C transmits 200 OK to the device 10-B (S56). Up to now, the device 10-B and the device 10-C are busy.

  The device 10-A transmits REFER Refer-To: device C to the device 10-D (S57). The device 10-D transmits 202 Accepted to the device 10-A (S58). The device 10-A transmits BYE to the device 10-D (S59). The device 10-D transmits 200 OK to the device 10-A (S61).

  The device 10-D transmits INVITE Referred-By: device A to the device 10-C (S62). At this time, undecided is described in the SDP of the message body. The device 10-C transmits 200 OK to the device 10-D (S63). At this time, undecided is described in the message body. The device 10-D transmits ACK to the device 10-C (S64). The device 10-D transmits NOTIFY to the device 10-A (S66). The device 10-A transmits 200 OK to the device 10-D (S67). Up to now, the device 10-D and the device 10-C are busy.

  The device 10-C transmits a control RTP to the device 10-B (S68). At this time, the control data is conference: request: server. The device 10-B transmits a control RTP to the device 10-C (S69). At this time, the control data is conference: response: client.

  With reference to FIG. 20, a processing flow of the video conference apparatus that leaves the base unit will be described. In FIG. 20, the communication control unit 102 determines whether to continue the multipoint video conference based on the setting from the user (S200). When it is determined that the multipoint connection video conference is to be continued (YES), the communication control unit 102 performs a disconnection process in order to disconnect the terminal that is the parent device from the multipoint connection video conference after leaving the device. (S201).

  The communication control unit 102 connects the SIP-URI of the transfer destination stored in the RAM 103 to the device that becomes the slave device after leaving the station and the device that operates as the parent device next, and the REFER message that is the SIP transfer message. Is designated as the transfer destination and transmitted (S202), and the process is terminated.

  On the other hand, when it is determined in step 200 that the multipoint connection video conference is not continued (NO), the communication control unit 102 performs the disconnection process on all the devices connected to the own device. (S203).

  In this way, when a multipoint connection video conference is to be continued in a situation where the base unit must leave during a multipoint connection video conference, a connection change process is newly performed for the device that becomes the base unit. Thus, the multipoint connection video conference can be continued.

  DESCRIPTION OF SYMBOLS 10 ... Multipoint connection video conference apparatus, 20 ... IP network, 30 ... In-SDP attribute description a setting correspondence table, 40 ... Control RTP setting correspondence table, 50 ... Transmission SDP information condition table by own device connection state at the time of outgoing call, 60 ... Operation condition table based on own device state and attribute description in received SDP, 100 ... multi-point connection video conference system, 101 ... line IF, 102 ... communication control unit, 103 ... RAM, 104 ... voice input IF, 105 ... voice compression unit , 106 ... audio expansion unit, 107 ... audio synthesis unit, 108 ... audio output IF, 109 ... video input IF, 110 ... video decoding unit, 111, 115 ... video scaling unit, 112 ... video compression unit, 113 ... video expansion unit , 114 ... Video composition unit, 116 ... Video encoding unit, 117 ... Video output IF, 301 ... SDP control code identifier, 3 2 ... SDP notification information, 401 ... RTP control code identifier, 402 ... RTP instruction code, 403 ... RTP notification information.

Claims (3)

A line IF for realizing input / output of data from the IP network, an audio input IF for taking in audio data from a microphone, an audio compression unit for compressing data from the audio input IF, and compressed audio Audio decompression unit for decompressing data, audio synthesis unit for synthesizing audio transmitted from multiple points, audio output IF unit for outputting audio data to a speaker, and capturing video data from a camera Video input IF, a video decoding unit for decoding data from the video input IF, a video scaling unit for scaling the decoded video data to a predetermined size, and a video compression for compressing the video data , Video decompression unit for decompressing compressed video data, and video composition unit for synthesizing video data transmitted from multiple points A video scaling unit for scaling the synthesized video to a predetermined size, a video encoding unit for encoding the scaled video, a video output IF for outputting the encoded video data to the TV, In a multipoint video conferencing apparatus comprising a RAM for storing SDP, RTP, and SIP information,
When another multipoint video conference apparatus participates in the video conference during the one-to-one video conference, the multipoint video conference is dynamically used by using the SDP and RTP received by the line IF. A multipoint videoconferencing apparatus comprising a communication control unit for determining a parent device in a system. The multipoint videoconferencing device according to claim 1,
When a teleconference is being performed as a one-to-two slave unit and a message for joining the teleconference is received from another multipoint video conference, the communication control unit transmits a transfer message including the SIP-URI of the master unit. Is returned to the other multipoint video conference. When a join message is received with the sender status being undetermined,
When the state is not connected, it is described in the response message that it is in an undetermined state, and is sent back to the transmission source.
When the state is undetermined state, it is described in the response message that it is the parent state, it is returned to the transmission source, the state is transitioned to the parent state,
When the status is the parent status, it is described in the response message that the status is the parent status, and is sent back to the sender.
When the status is a child status, a redirect message is returned to the sender,
When a join message with a parent status is received,
When the state is not connected, it is described in the response message that it is a child state, it is returned to the transmission source, the state is transitioned to a child state,
A multi-point connection video conference apparatus, wherein a busy message is returned to the transmission source when the state is any of an undecided state, a parent state, and a child state.

Images