JP2007318556A - Multipoint video conference system in communication network - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multipoint video conference system by several terminals only in an IP network which is not affected by deterioration of image/sound quality due to increase in number of terminals participating in the conference. <P>SOLUTION: An annular communication path is formed on the IP network 10 by connecting terminals A20A-D20D with one another. Each of the terminals 20 is connected with different two terminals, displays/outputs image/sound data which are received from one of the two connected terminals on its own terminal as well as updates image/sound data according to information of control packets for conference control transmitted by the terminal A20A as a chairman terminal, and relays the new image/sound data and the control packets to the other terminal so that they may be orbited along the ring. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention transmits video / audio data on a communication network based on connection control information transmitted on a communication network such as an IP network, and a video conference terminal (hereinafter referred to as a video phone / conference terminal) connected to the communication network. In particular, the present invention relates to a multipoint video conferencing system in which conversations and conferences are performed by connecting three or more multipoint terminals.

  As a multipoint video conference system in a conventional IP network, there is a multipoint video conference system using a center device such as a multipoint control device (hereinafter, MCU) and various servers. In this mode, all terminals participating in the multipoint video conference are connected to the center apparatus, and the center apparatus performs conference control such as terminal address conversion / transmission band management and multiplexing / demultiplexing / combining of image / audio data. Thus, a multipoint video conference is realized (for example, see Patent Document 1).

  There is also a multipoint video conference system using an IP multicast function. In this mode, all terminals participating in the multipoint video conference have means for knowing the IP addresses for multipoint distribution used by the own terminal and all other terminals, and for multipoint distribution assigned to the own terminal. Video / audio data is transmitted to all other terminals using the IP address, and similarly, video / audio transmitted from each terminal using the IP address for multi-point distribution assigned to all other terminals. A multipoint conference is realized by receiving data (see, for example, Patent Document 2).

Furthermore, as a multipoint video conference system other than the IP network, there is a form in which terminals are connected in a ring shape in ISDN. In this mode, each terminal participating in the multipoint video conference connects the two B channels of the ISDN interface to different terminals, so that all terminals participating in the multipoint video conference are connected in a ring shape and controlled. A multipoint video conference is realized by sequentially relaying data and video / audio data (see, for example, Patent Document 3).
JP 2002-374509 A JP 2006-005526 A Japanese Patent Laid-Open No. 5-22322

  In the prior art disclosed in Patent Document 1, a center device such as an MCU is provided and a multipoint video conference is realized by connecting the center device and all terminals participating in the multipoint video conference. Depending on the ability, the number of terminals that can participate in a multipoint video conference may be limited, and the video / audio quality of the conference may deteriorate. Even if all the terminals participating in the multipoint video conference are in a state where the conference can be performed, if some or all of the center devices are not operating for some reason, or if the multipoint video conference group is already in use, If the capacity of the device has reached its limit, a multipoint video conference by a new group cannot be held.

  In the prior art disclosed in Patent Document 2, all the terminals participating in the multipoint video conference use the multipoint distribution IP address to transmit and receive video / audio data to realize the multipoint video conference. It is necessary to process packets sent using multi-point distribution IP addresses for the number of terminals participating in the multi-point video conference, and each terminal receives packets as the number of terminals participating in the multi-point video conference increases. The amount of processing increases, and the video / audio quality of the conference may deteriorate. In addition, since the network conditions to which each terminal participating in the multipoint video conference belongs is limited to a network that can use the multicast function, a terminal that belongs to a network that does not satisfy the conditions even though it is an IP network is in a multipoint video conference. You will not be able to participate.

  The prior art disclosed in Patent Document 3 has no problem with the above-mentioned problems occurring in Patent Document 1 and Patent Document 2, but cannot be applied to a multipoint video conference in an IP network because the network is limited to ISDN. .

  An object of the present invention is to provide a multipoint video conference system that can realize a multipoint video conference by using only terminals participating in the multipoint video conference without requiring a center device such as MCU in an IP network. Another object of the present invention is to provide a multi-point video conference system that can realize a multi-point video conference without being affected by video / audio quality degradation due to an increase in the number of terminals participating in the multi-point video conference in an IP network. There is to do.

  To achieve the above object, the present invention provides a communication network, for example, an IP network, in which all terminals participating in a multipoint video conference are connected to a ring-shaped communication path, and at least one terminal proceeds with the conference. Control means for setting and sending control data, sending the control data and video / audio data on the communication path, and other terminals sending their own video and audio data of the video / audio data by the control data By replacing it and sending it to the ring-shaped communication path, a multipoint video conference can be realized without requiring a center device such as MCU.

  In order to achieve the other object, all terminals participating in a multi-point video conference in a communication network, for example, an IP network, need only connect to two terminals at the maximum, from one of the two terminals. By providing the means to capture the received control data and video / audio data to the terminal at the same time as relaying to the other terminal, the video / audio quality is degraded even when the number of terminals participating in the multipoint video conference increases. Multipoint video conferencing is now possible.

  The multipoint video conference system according to the present invention connects a plurality of video conference terminals to a communication network, connects each of the plurality of video conference terminals to two different video conference terminals, and controls connection between the video conference terminals. The communication path between the plurality of video conference terminals is connected in a ring shape by the connection control information for performing the multipoint video conference by the conference control information for designating the video conference terminal for transmitting the video and designating the display screen. I tried to control it.

  The multipoint video conference system according to the present invention is the multipoint video conference system, wherein at least one of the plurality of video conference terminals includes the plurality of video conference terminals in a ring shape. Connection control information setting means for setting connection control information for performing connection control between each video conference terminal so as to connect to a multi-point video conference for specifying a video conference terminal for transmitting video and specifying the configuration of its display screen Conference control information setting means for setting conference control information for controlling the connection, control for transmitting the connection control information set by the connection control information setting means and the conference control information set by the conference control information setting means as control data A video transmission terminal configured to set video / audio data based on the conference information and set in the connection control information; And a video-audio data transmitting means for transmitting image and audio data.

  Furthermore, the multipoint video conference system according to the present invention is the multipoint video conference system, wherein each of the video conference terminals includes a specific video conference terminal set in the connection control information in the received control data. Connection control means for controlling connection and disconnection of the video, video conference terminal for transmitting video specified in the conference control information in the control data, and video data for recognizing the configuration of the display screen and configuring the display screen of the terminal itself Control means and control data relay means for relaying control data received so that the control data circulates on a loop constituted by the plurality of video conference terminals.

  In addition, the multi-point video conference system according to the present invention is the multi-point video conference system, wherein each video conference terminal has its own video / audio data sent from another video conference terminal. Video / audio data updating means for updating the video / audio data to new video / audio data and sending it to another video conference terminal is provided. Furthermore, the multipoint video conference system according to the present invention is the multipoint video conference system described above, wherein the communication network is an IP network.

  According to the present invention, it is possible to provide a multipoint video conference system capable of realizing a multipoint video conference by using only a terminal without requiring a center device such as MCU in the IP network.

  That is, according to the present invention, an expensive center device such as an MCU is not required in an IP network, and a multipoint video conference can be realized only with a relatively inexpensive terminal, which can contribute to the spread of the multipoint video conference system. It can also contribute to the acquisition of new network users by network operators.

  An embodiment of a multipoint video conference system in an IP network according to the present invention will be described below.

  The configuration of a multipoint video conference system in a communication network according to an embodiment of the present invention will be described using FIG. 1 assuming that the number of terminals participating in the multipoint video conference is four. The multipoint video conference system according to the present invention is configured by connecting a plurality of terminals 20 </ b> A to 20 </ b> D so as to form a ring-shaped communication path on the IP network 10. In the present invention, the communication network may be a communication network in which control data for controlling connection of a plurality of video conference terminals and video / audio data of the plurality of video conference terminals are transmitted and received on a ring-shaped communication path. For example, an IP network is a typical example. An IP network 10 in the figure is an IP network based on a local area network (LAN) or a wide area network (WAN) such as a public Internet service.

  Terminal A 20A, terminal B 20B, terminal C 20C, and terminal D 20D are video conference terminals that have means capable of connecting to a plurality of terminals via the IP network 10 and participate in a multipoint video conference. In addition, the terminal A20A is a chair terminal that plays a role as the chair of the multipoint video conference in the present embodiment. In this embodiment, the terminals A20A to D20D participating in the multipoint video conference are connected to a ring-shaped communication path on the IP network. That is, terminal A20A connects to terminal B20B and terminal D20D using two different IP connections, similarly terminal B20B is terminal C20C and terminal A20A, terminal C20C is terminal D20D and terminal B20B, and terminal D20D is terminal A20A and A ring-shaped communication path is configured by connecting to the terminal C20C. This connection is made by connection control information transmitted and received on the network 10. Each of the terminals 20A to 20D transmits / receives video / audio data on a ring-shaped communication path. Each of the terminals 20A to 20D updates the video / audio data of the own terminal of the received video / audio data based on the conference control data transmitted from the chair terminal A20A to the ring-shaped communication path.

  The configuration of the video conference terminal according to the present embodiment will be described using the terminal A 20A as an example with reference to FIG. In the present embodiment, the same configuration can be applied to terminals other than the chairman terminal. The video conference terminal 20A includes a video input / output unit 21, a video codec unit 22, a video synthesis / division processing unit 23, an audio input / output unit 24, an audio codec unit 25, an audio addition / subtraction processing unit 26, The apparatus includes a separation unit 27, a system control unit 28, and a network interface unit 29. In FIG. 2, solid lines between the units indicate connection of data lines for video / audio data and control data, and dotted lines indicate connection of control signal lines such as a CPU bus by the CPU in the system control unit 28.

  The video input / output unit 21 is a connection interface between a camera and a video monitor. In the case of a videophone type terminal, a built-in camera and a built-in video monitor are also included. The video codec unit 22 is a function / means for performing encoding / decoding processing of a video signal. The video synthesizing / dividing processing unit 23 is a function / means for synthesizing / dividing video encoded data received from another terminal and video encoded data of the terminal itself. It functions as a video data updating means that is replaced and sent to another video conference terminal.

  The voice input / output unit 24 is a connection interface between a microphone and a speaker. In the case of a videophone type terminal, a built-in microphone and a built-in speaker are also included. The audio codec unit 25 is a function / means for performing encoding / decoding processing of an audio signal. The voice addition / subtraction processing unit 26 is a function / means for performing addition / subtraction processing of voice coded data received from another terminal and the voice coded data of the own terminal. The voice addition / subtraction processing unit 26 converts own voice data into new voice based on the conference control data. It functions as an audio data updating means that replaces data and sends it to other video conference terminals.

  The multiplexing / demultiplexing unit 27 is a function / means for performing multiplexing processing of video / audio data to be output by the terminal itself and separation processing of video / audio multiplexed data received from other terminals. The network interface unit 29 is a lower layer interface with the IP network 10.

  The system control unit 28 is a function / means for controlling each unit in the terminal, processing packet processing of video / audio encoded data described later, and managing an address book described later in which members of the multipoint video conference are registered. The system control unit 28 also performs control data processing, which will be described later, and processing of various protocols including IP in communication with the IP network 10. Furthermore, the system control unit 28 stores storage means (function) for storing control data such as terminal information, connection information, and conference information of the address book and video conference participation terminals, and connections received from terminals on the upstream side of the ring-shaped communication path. It has a control data relay function for relaying control data composed of control information and conference control information to a terminal on the downstream side. Further, the system control unit 28 includes a connection control information setting function for setting connection control information for performing connection control between the video conference terminals so as to connect a plurality of video conference terminals in a ring shape, and a video conference for transmitting video. Conference control information setting function for setting conference control information for controlling a multipoint video conference for specifying a terminal and specifying a display screen configuration thereof, and connection control information and conference control information setting function set by the connection control information setting function Control data transmission function to transmit the conference control information set by control as control data, and video to send video / audio data to the video conference terminal set in the connection control information by setting video / audio data based on the conference information・ Has a voice data transmission function.

  Further, the system control unit 28 transmits a connection control function for controlling connection and disconnection with a specific terminal set in the connection control information in the control data, and an image specified in the conference control information in the control data. It has a video data control function for recognizing the configuration of the terminal and its display screen and configuring the display screen of the terminal itself.

  Next, a data flow from input of video / audio signals to transmission to the IP network 10 will be described. The video signal input from the video input / output unit 21 is video-encoded by the video codec unit 22, and the video signal is recorded in the video synthesizing / dividing processing unit 23 after the previous video is deleted from the video data from the upstream side. The video is transferred to the multiplexing / demultiplexing unit 27 after being subjected to the process of combining the video with the other terminal video. The audio signal input from the audio input / output unit 24 is encoded by the audio codec unit 25, and the local terminal audio is converted into the trace of the previous local terminal audio deleted from the audio data from the upstream side by the audio addition / subtraction processing unit 26. After the addition process, the data is transferred to the multiplexing / separating unit 27. For video / audio encoding, MPEG2, MPEG4, H. 263, H.M. H.264, etc. 711, G.G. There are various schemes such as H.723, but this embodiment is not particularly limited as long as it is a coding scheme capable of implementing the present invention.

  The multiplexing / separating unit 27 multiplexes the video data and audio data transferred from the video synthesizing / dividing processing unit 23 and the audio addition / subtraction processing unit 26 into frames or packets suitable for real-time communication, and transfers them to the system control unit 28. In the present embodiment, as an example of the frame or packet, an RTP packet conforming to RFC1889, 1890, etc. published by IETF (Internet Engineering Task Force), which is generally used well, is assumed. The frame or packet is not limited to the above method, and may be any frame or packet that can implement the present invention, and is not particularly defined.

  The system control unit 28 adds a UDP / IP header to the video / audio multiplexed data transferred from the multiplexing / demultiplexing unit 27, and also adds a TCP / IP header to the control data to be described later to add a network interface unit 29. Forward to. The network interface unit 29 further adds a lower layer header to the video / audio data and the control data, and sends them to the IP network 10.

  The video / audio data received from the IP network 10 is output from the video input / output unit 21 and the audio input / output unit 24 by a transfer procedure completely opposite to the data flow, and the received control data is a system control unit. After confirmation at 28, transmission processing is performed for the terminal to be transferred. The system control unit 28 compares the received control data with the previous control data stored in the storage means to determine whether there is a change. If there is a change, the system control unit 28 updates the address book or control data.

  Hereinafter, a means for realizing video display and audio output in each terminal will be described in detail. First, regarding the video display, in this system, the chairman terminal A20A forms the display screen division number constituting the conference control information and the screen display position where the video of each terminal is to be displayed in a ring shape by a control packet described later. Send to the communication path to notify all terminals participating in the multipoint video conference.

  An example of the screen division image and the screen display position will be described with reference to FIG. When the display screen division number is “1”, an image for one terminal is displayed on the entire screen as shown in FIG. 3A. Similarly, when the display screen division number is “2”, FIG. When the display screen division number is “4”, the display screen is divided into four as shown in FIG. In FIG. 3, the screen display positions are shown as “screen display position 1”, “screen display position 2”, “screen display position 3”, and “screen display position 4”, respectively. For example, according to the rules defined by the video encoding method used in this system, each terminal uses only the area allocated for the video of its own terminal among the video data of the full screen. To the next terminal after combining with another area again.

  For example, when the number of display screen divisions is “4”, the terminal whose screen display position 3 is specified by the control packet includes the video data received from the multiplexing / separating unit 27 in the video composition / division processing unit 23 as shown in FIG. The data in the area corresponding to “screen display position 3” in (c) is replaced with the latest video data of its own terminal, and is synthesized again with the video data of the other area as it is, and then returned to the multiplexing / demultiplexing unit 27. At the same time, for video to be displayed on its own terminal, the video data received from the multiplexing / demultiplexing unit 27 is directly decoded by the video codec unit 22 and output from the video input / output unit 21. As a result of the above process, a video as shown in FIG. 3C in which the previous video of the terminal is displayed at “screen display position 3” is displayed on the screen of the terminal, and “screen display” is displayed on the screen of the other terminal. A video as shown in FIG. 3C in which the area corresponding to the position 3 ″ is updated to the latest video of the terminal itself is transferred and displayed.

  For audio output, the audio addition / subtraction processing unit 26 subtracts only the audio data of the own terminal previously transmitted from the audio data received from the multiplexing / demultiplexing unit 27, and a new own terminal is added to the difference component of the audio data. Are added back to the multiplexing / demultiplexing unit 27. At the same time, for the audio to be output to the own terminal, the audio data (difference component) in a state where only the subtraction of the audio data of the own terminal previously transmitted is performed on the audio data received from the multiplexing / demultiplexing unit 27. Is decoded by the audio codec unit 25 and output from the audio input / output unit 24. Through the above process, the voices of all other terminals are output to the own terminal, and the voices of all terminals including the latest voice of the own terminal are transferred to the other terminals and output.

  The image of the address book managed by each terminal will be described with reference to FIG. The address book is described in association with terminal names and IP addresses. Further, the address book can describe the connection status of each terminal, the name of the site, and the like. The address book is managed by the system control unit 28 of each terminal, and is set by either registration by user input or automatic registration of terminal information with a connection record. In the present embodiment, the terminal A 20A is the chair terminal of the multipoint video conference, and it is necessary to register terminal information such as IP addresses of all the terminals participating in the conference in the address book in advance. Also, the conference participants using the terminal B20B, the terminal C20C, and the terminal D20D need to obtain terminal information such as the IP address of the terminal A20A that is the chairman terminal in advance.

  An image of the control data CD used in the multipoint video conference system will be described with reference to FIG. Since connection control information and conference control information are stored in the control data CD and it is necessary to reliably reach all terminals participating in the conference, retransmission / retransmission that can avoid data error / loss / unreachability It is necessary to send and receive in a procedure that can use functions such as request and reception confirmation notification. The control data CD in the present embodiment is realized by, for example, a TCP packet conforming to RFC793 released by IETF, and is hereinafter expressed as a control packet CP. The control packet CP is a data area following the lower layer header and the TCP header, as a common information, a route confirmation flag indicating whether the packet is a control packet used for route confirmation, a terminal currently participating in a multipoint video conference The number of participating terminals indicating the number, the video / audio communication port indicating the port number used for video / audio communication, and the display screen division number indicating the screen division number of the video displayed on the video monitor of each terminal are sequentially stored. In the area following the common information, as terminal information of the terminal currently participating in the multipoint video conference, the IP address of the corresponding terminal, the terminal state indicating the connection state of the terminal, and the terminal screen indicating the screen display position of the terminal Stores the display position. The terminal information is stored for the number of terminals participating in the conference, and each terminal performs connection control so as to form a ring-shaped connection in the order in which the terminal information is stored. The terminal IP address of the common information and terminal information of the control packet CP can be used as connection control information, and the terminal state and screen display position of the terminal information can be used as conference control information.

  In the present embodiment, the terminal A 20A serves as the chairman terminal and generates a control packet CP as shown in FIG. 5 by the system control unit 28 and transmits it to the next terminal (terminal B 20B). Also, the terminal B 20B, the terminal C 20C, and the terminal D 20D each receive the control packet CP transmitted from the terminal A 20A and sequentially relay it to the next terminal, so that each terminal participating in the multipoint video conference Information can be shared.

  The outline of the operation of the multipoint video conference system in the IP network according to the present embodiment will be described below.

  First, the operation when terminal B20B joins the conference as the first terminal with respect to terminal A20A, which is the chairman terminal, will be described using the sequence diagram of FIG. The terminal B20B transmits a conference participation request in a format predetermined by the multipoint video conference system to the IP address of the terminal A20A, which is the chairman terminal acquired in advance (S1).

  Upon receiving the conference participation request, the terminal A 20A confirms that the terminal B 20B is registered by using an address book as shown in FIG. 4 created in advance (S2), and permits conference participation to the IP address of the terminal B 20B ( Connection destination information) is transmitted (S3). In this conference permission (connection destination information), terminal information including the port number used for communication of the control packet CP in the multipoint video conference and the IP address of the terminal to which video / audio data should be communicated is stored. The terminal B 20B can know the IP address of the terminal to communicate with.

  The terminal A20A transmits a control packet 1 (route confirmation) as shown in FIG. 7A following the conference permission (connection destination information) (S4). In the example of FIG. 6, since the terminal participating in the conference is only the terminal B 20B, the terminal A 20A transmits the control packet 1 to the IP address of the terminal B 20B. Further, since this control packet 1 is used for route confirmation, the route confirmation flag is “1”, the number of participating terminals “1”, the video / audio communication port “m”, the display image division number “1”, the terminal A IP address “AA”, terminal A state “conference”, terminal A screen display position “1”, terminal B IP address “BB”, terminal B state “registering”, terminal B screen display position “0” As sent. At this time, video / audio communication has not yet started between the terminal A and the terminal B.

  The terminal B 20B receives the control packet 1 to transmit the route confirmation control packet 1, the current conference participation terminal number is “1”, and the port number for video / audio communication is “m”. ”, The current number of display screen divisions is“ 1 ”, and the terminal that should communicate is the terminal A 20A and the IP address is“ AA ”. Further, although the terminal information of the terminal B 20B itself is stored in the control packet 1 (route confirmation), the state is “registering” and the screen display position is “0 (no display position)”. Based on this information, the terminal B 20B transmits a connection request addressed to the terminal A 20A (S5), and the terminal A 20A receiving the connection request transmits a connection request response addressed to the terminal B 20B (S6). When the terminal B 20B receives this connection request response, a call with the terminal A 20A is established. Regarding call control, in addition to the above procedure, for example, ITU-T (International Telecommunication Union-Telecommunication Sector) has been standardized. It can also be established using a call control protocol such as H.323.

  The terminal B20B transfers the control packet 1 (route confirmation) received to the terminal A20A with the establishment of the call (S7), and the terminal A20A returns the terminal when the control packet 1 (route confirmation) returns. It can be recognized that the call of B20B has been established.

  The terminal A20A transmits the control packet 2 as shown in FIG. 7B to the terminal B20B when the period a elapses after receiving the control packet 1 (S8), and then video is transmitted to the terminal B20B. -Voice communication is started (S9), (S12). The terminal B20B acquires the information of the received control packet 2, updates the common information and the terminal information stored in the storage unit, and then transfers the received control packet 2 to the terminal A20A (S10). In the control packet 2, the route confirmation flag in the control packet 1 is “0”, the number of conference participation terminals is “2”, the display screen division number is “2”, and the status of the terminal B 20B is “conference”. The display position is updated to “2”. The terminal B 20B performs video / audio communication by transmitting video / audio data obtained by superimposing its own audio data and video data to the video / audio data received in step S9 to the terminal A 20A.

  When the period a further elapses, the terminal A 20A transmits the control packet 3 corresponding to the presence / absence of the state change (S11), and then the terminal B 20B transfers the received control packet 3 to the terminal A 20A. (S13). Thereby, video / audio communication is performed between the terminal A20A and the terminal B20B. Thereafter, communication using the control packet CP is performed every period a until the multipoint video conference is completed asynchronously with the video / audio communication.

  In this way, a multipoint video conference can be realized only by terminals participating in the multipoint video conference without requiring a center device such as MCU in the IP network.

  Next, the operation when terminal D20D participates in the middle of a multipoint video conference in which three terminals A20A, terminal B20B, and terminal C20C are connected in a ring shape on the IP network is shown in FIG. This will be described with reference to a sequence diagram. Terminals A20A, B20B, and C20C are connected in a ring shape on the IP network and have already been in a conference, and are in a video / audio communication state (S21). At the same time, the conference control by the control packet CP is also performed, the terminal A20A as the chairman terminal transmits the control packet n (S22), the terminal B20B transfers the control packet n to the terminal C20C (S23), and the terminal C20C is in the state of transferring control packet n to terminal A20A (S24).

  The terminal D20D who wants to participate in the multipoint video conference midway transmits a conference participation request to the IP address of the terminal A20A which is the chairman terminal acquired in advance (S25). Upon receiving the conference participation request, the terminal A 20A confirms that the terminal D 20D is registered by using an address book as shown in FIG. 4 created in advance (S26), and permits conference participation to the IP address of the terminal D 20D ( Connection destination information) is transmitted (S27). The terminal D20D can know the terminal information including the port number for control packet communication in the multipoint video conference and the IP address of the terminal to which video / audio data should be communicated by this conference participation permission.

  The terminal A20A transmits a control packet n + 1 (route confirmation) for route confirmation as shown in FIG. 9B to the terminal B20B when transmitting a control packet following permission to join the conference (S28). The terminal B20B recognizes that the terminal D20D is being registered by the control packet n + 1 and that the connection state of the terminal D20D is not changed, and stores the new terminal D20D being registered in the storage means, and then sends it to the terminal C20C. The control packet n + 1 (route confirmation) is transferred (S29). The terminal C20C recognizes that the terminal D20D is being registered by the control packet n + 1 and that the transmission destination of the subsequent video / audio data and control packet is changed from the terminal A20A to the terminal D20D, and a new terminal is stored in the storage means. After storing D20D being registered, the control packet n + 1 (route confirmation) is transferred to the terminal D20D (S30).

  The terminal D20D indicates that the current number of conference participation terminals is “3” and the port number for video / audio communication is “m” based on the control packet n + 1 (route confirmation) received from the upstream terminal C20C. The display screen division number is “3”, the terminal that should receive the video / audio data is the terminal C20C and the IP address is “CC”, and the terminal that should transmit the video / audio data is the terminal A20A. And know that the IP address is “AA”. Further, although the terminal information of the terminal D20D itself is stored in the control packet n + 1 (route confirmation), the state is “registering” and the screen display position is “0 (no display position)”. Based on the information, the terminal D20D transmits a connection request C to the terminal C20C (S31), and transmits a connection request A to the terminal A20A (S32). The terminal C20C that has received the connection request C transmits a connection request response C to the terminal D20D (S33), and the terminal A20A that has received the connection request A transmits a connection request response A to the terminal D20D (S34). When terminal D20D receives these connection request responses, a call with terminal C20C and terminal A20A is established.

  The terminal D20D transfers the control packet n + 1 (route confirmation) received in step S30 to the terminal A20A with the establishment of the call (S35). The terminal A20A can recognize that the call of the terminal D20D has been established by the return of the control packet n + 1 (route confirmation), and after the period a has elapsed, the common information as shown in FIG. The route confirmation flag is updated to “0”, the number of participating terminals is “4”, the number of display screen divisions is updated to “4”, the status of the terminal D information is “in meeting”, and the screen display position is updated to “4”. The control packet n + 2 is transmitted to the terminal B 20B (S36).

  The terminal B 20B confirms that the number of conference participation terminals, the number of display screen divisions, the screen display position, and the state of the terminal D 20D are updated by the control packet n + 2, and stores the information stored in the storage unit of the system control unit 28. The control packet n + 2 is updated and transferred to the next terminal C20C (S37), and the terminal C20C updates the number of conference participation terminals, the number of display screen divisions, the screen display position, and the state of the terminal D20D by the control packet n + 2. The information stored in the storage means of the system control unit 28 is updated, and the control packet n + 2 is transferred to the next terminal D20D (S38). The terminal D20D uses the control packet n + 2 to determine the number of conference participation terminals, Confirmed that the number of display screen divisions, the screen display position and state of the terminal itself have been updated, and stored in the storage means of the system control unit 28 Update the broadcast, and transfers the control packet n + 2 to the next terminal A20A (S39). New video / audio communications (S40), (S41), (S42), and (S47) are started immediately after the terminal A 20A transmits the control packet n + 2.

  When the period a further elapses, the terminal A 20A transmits a control packet n + 3 (S43), the terminal B 20B transfers the control packet n + 3 to the next terminal C 20C (S44), and the terminal C 20C transmits the control packet n + 3 next. Is transferred to the terminal D20D (S45), and the terminal D20D transfers the control packet n + 3 to the next terminal A20A (S46). Thereafter, communication using control packets (S43 to S46) is performed every period a until the multipoint video conference is completed.

  As described above, the terminal D20D who wishes to participate in the multipoint video conference using the communication path formed in a ring shape on the IP network can perform other operations without destroying the communication path formed in the ring shape. It is possible to participate in a multi-point video conference without inconvenience to the terminal.

  The operation when the terminal C 20C becomes unable to communicate for some reason in the multipoint video conference by the four terminals configured by the sequence diagram of FIG. 8 will be described with reference to the sequence diagram of FIG. Terminals A20A, B20B, C20C, and D20D are connected in a ring shape on the IP network to conduct a multipoint video conference, and are in a state where video / audio communication is being performed (S51). At the same time, the conference control by the control packet is also performed, and the terminal A 20A as the chairman terminal transmits a control packet m as shown in FIG. 11A to the terminal B 20B (S52), and the terminal B 20B m is transferred to the next terminal C20C (S53), the terminal C20C transfers the control packet m to the next terminal D20D (S54), and the terminal D20D is transferring the control packet m to the terminal A20A (S43). Is in a state. Video / audio communication is performed from the terminal A20A to the terminal B20B (S56), and the terminal B20B replaces its own video data transmitted last time in the video data with new video data and also transmits its own audio data previously transmitted. Video / audio communication is performed by transmitting new video / audio data in which is replaced with new audio data to the terminal C20C (S58).

  Assume that after the transfer of the control packet m is completed, the terminal C20C falls into an abnormal state (S57) for some reason and communication becomes impossible. The terminal B20B normally performs video / audio communication with the terminal A20A (S56) and sequentially relays the video / audio data received from the terminal A20A to the terminal C20C, but the terminal C20C cannot communicate with the terminal A20A. In this state, the video / audio communication (S58) cannot be normally performed, and the ring configuration of the multipoint video conference is broken.

  In this state, since the terminal A 20A cannot know that the terminal C 20C is in a communication disabled state, the terminal A 20A transmits a control packet m + 1 to the terminal B 20B to continue the conference (S59). Since the terminal B 20B cannot know that the terminal C 20C is in an incommunicable state at this time, the control packet m + 1 is transferred to the terminal C 20C (S60), but the reception confirmation notification in TCP is not returned from the terminal C 20C. Then, the abnormal state of the terminal C20C is recognized (S61), and the terminal abnormality report C is transmitted to the terminal A20A as the chairman terminal (S62). Upon receiving the terminal abnormality report C, the terminal A 20A recognizes that the terminal C 20C is in an abnormal state and the ring configuration of the multipoint video conference has failed, and immediately creates a new ring configuration as shown in FIG. A route confirmation control packet m + 2 (route confirmation) in which the route confirmation flag is “1” and the state of the terminal C is “abnormal” is transmitted to the terminal B 20B (S63).

  The terminal B20B recognizes that the transmission destination of the subsequent video / audio data and control packet is changed from the terminal C20C to the terminal D20D by the control packet m + 2, rewrites the information in the storage means, and then sends the control packet to the terminal D20D. m + 2 is transferred (S64). The terminal D20D recognizes that the transmission source of the subsequent video / audio data and control packet is changed from the terminal C20C to the terminal B20B by the control packet m + 2, rewrites the information in the storage means, and then connects to the terminal B20B. Request B is transmitted (S65). The terminal B20B that has received the connection request B transmits a connection request response B to the terminal D20D (S66). When the terminal D20D receives the connection request response B, a call between the terminal B20B and the terminal D20D is established. The

  The terminal D20D transfers the control packet m + 2 received in step S64 to the terminal A20A with the establishment of the call (S67). The terminal A20A can recognize that a call for configuring a new ring state excluding the terminal C20C has been established by the return of the control packet m + 2, and the common information as shown in FIG. Route confirmation flag “0”, number of participating terminals “3”, display screen division number “3”, terminal C information is deleted, and control packet m + 3 with the screen display position “3” of terminal D information addressed to terminal B 20B (S68). The terminal B20B confirms that the number of terminals participating in the conference and the number of display screen divisions has been updated by the control packet m + 3 and the terminal information of the terminal C20C has been deleted, and transfers the control packet m + 3 to the terminal D20D (S69). . The terminal D20D confirms that the number of conference participation terminals, the number of display screen divisions, and the screen display position are updated by the packet m + 3, and that the terminal information of the terminal C20C is deleted. The packet m + 3 is transferred to the terminal A 20A (S70). New video / audio communication between terminal A and terminal B, between terminal B and terminal D, and between terminal D and terminal A is started immediately after terminal A 20A transmits control packet m + 3 (S71), (S72). (S73). When the period a further elapses, the terminal A 20A transmits a control packet m + 4 similar to the control packet m + 3 to the terminal B 20B (S74), and the terminal B 20B transfers the control packet m + 4 to the terminal D 20D (S75). D20D transfers control packet m + 3 to terminal A20A (S76).

  In this way, when a failure occurs in any of the terminals that are performing a point video conference connected in a ring shape on the IP network, the ring-shaped communication path is prevented from being destroyed for a long time. Thus, it is possible to form a ring-shaped communication path excluding the terminal in which the failure occurs and to continue the multipoint video conference.

  If the terminal in the abnormal state is the terminal B20B, the terminal A20A, which is the chair terminal, can directly know that there is no control packet reception confirmation notification, so the terminal A20A can immediately control for route confirmation. The packet is transmitted to the terminal C20C, and the terminal A20A returns a connection request response to the connection request from the terminal C20C. When the terminal D20D enters an abnormal state, a terminal abnormality report is transmitted from the terminal C20C to the terminal A20A, and the terminal C20C returns a connection request response to the connection request from the terminal A20A.

  Further, the operation when the terminal C20C intentionally leaves in the middle of a multipoint video conference with four terminals configured by the sequence diagram of FIG. 8 will be described with reference to the sequence diagram of FIG. Terminal A 20A, terminal B 20B, terminal C 20C, and terminal D 20D are connected in a ring shape on the IP network to conduct a multipoint video conference and are in a video / audio communication state (S81). At the same time, the conference control is also performed by the control packet, and the terminal A 20A as the chairman terminal transmits a control packet k as shown in FIG. 13A to the terminal B 20B (S82), and the terminal B 20B Is transferred to the terminal C20C (S83), the terminal C20C transfers the control packet k to the terminal D20D (S84), and the terminal D20D is in a state of transferring the control packet k to the terminal A20A (S85).

  Here, the terminal C20C who wants to leave midway transmits a meeting leaving request to the terminal A20A which is the chairman terminal (S86). The terminal A20A that has received the request to leave the conference has a route in which the route confirmation flag of the common information as shown in FIG. 13B is “1” and the state of the terminal C information is “leaved” in the next control packet transmission cycle. The control packet k + 1 for confirmation (route confirmation) is transmitted to the terminal B 20B (S87). The terminal B20B recognizes that the terminal C20C is away from the control packet k + 1, and the transmission destination of the subsequent video / audio data and control packet is changed from the terminal C20C to the terminal D20D, and updates the information in the storage unit. The control packet k + 1 is transferred to the terminal C20C (S88). The terminal C20C is a route confirmation control packet for the leaving request transmitted by the own terminal by the control packet k + 1, recognizes that the information of the own terminal is “leaving away”, and sends the control packet k + 1 to the terminal D20D. Is transferred (S89). The terminal D20D recognizes that the terminal C20C is away from the control packet k + 1 and the transmission source of the subsequent video / audio data and control packet is changed from the terminal C20C to the terminal B20B, and updates the information in the storage unit. The connection request B is transmitted to the terminal B 20B (S90). The terminal B 20B that has received the connection request B transmits a connection request response B to the terminal D 20D (S91). When the terminal D 20D receives the connection request response B, a call between the terminal B 20B and the terminal D 20D is established. The

  The terminal D20D transfers the control packet k + 1 received in step S89 to the terminal A20A with the establishment of the call (S92). The terminal A20A can recognize that the call for composing a new ring state after the terminal C20C leaves due to the return of the control packet k + 1 has been established, and immediately grants permission to leave the conference to the terminal C20C. Then, the terminal C20C leaves the multipoint video conference (S94). The terminal A20A sets the common information route confirmation flag “0” and the display division number “3” as shown in FIG. 13C, deletes the terminal C information, and sets the screen display position “3” of the terminal D information. The control packet k + 2 is transmitted to the terminal B 20B (S95). The terminal B 20B confirms that the number of terminals participating in the conference and the number of display screen divisions are updated by the control packet k + 2, the terminal information state of the terminal C 20C is deleted, updates the information in the storage means, and then receives the control packet k + 2 is transferred to the terminal D20D (S96). After the terminal D20D confirms that the number of conference participation terminals, the number of display screen divisions, and the screen display position are updated by the packet k + 2, the terminal information state of the terminal C20C is deleted, and updates the information in the storage unit Then, the control packet k + 2 is transferred to the terminal A20A (S97). New video / audio communication between the terminal A20A and the terminal B20B, between the terminal B20B and the terminal D20D, and between the terminal D20D and the terminal A20A is started immediately after the terminal A20A transmits the control packet k + 2 (S98), (S99). (S100). When the period a further elapses, the terminal A 20A transmits a control packet k + 3 similar to the control packet k + 2 to the terminal B 20B (S101), and the terminal B 20B transfers the control packet k + 3 to the terminal D 20D (S102). D20D transfers the control packet k + 3 to terminal A20A (S103).

  In this way, the terminal C20C participating in the multipoint videoconference on the communication path connected in a ring shape with the IP network leaves the multipoint videoconference at any time without breaking the ring-shaped communication path. can do.

The block diagram of the multipoint video conference system of a present Example. FIG. 2 is a block diagram of a videophone / conference terminal according to the present embodiment. The screen division | segmentation image figure of a present Example. The address book image figure of a present Example. The control data image figure of a present Example. The meeting start operation | movement sequence diagram by 2 terminals of a present Example. The control packet image figure at the time of the meeting start of a present Example. FIG. 4 is a sequence diagram of an operation for joining a conference in the present embodiment. The control packet image figure at the time of participating in the middle of a meeting of a present Example. The operation | movement sequence diagram at the time of terminal abnormality generation of a present Example. The control packet image figure at the time of terminal abnormality generation of a present Example. FIG. 6 is a sequence diagram of an operation for leaving a meeting from a meeting according to the present embodiment. The control packet image figure at the time of leaving a meeting in the middle of a present Example.

Explanation of symbols

10 IP Network 20A-20B Videophone / Conference Terminals AD
21 video input / output unit 22 video codec unit 23 video synthesis / division processing unit 24 audio input / output unit 25 audio codec unit 26 audio addition / subtraction processing unit 27 multiplexing / separation unit 28 system control unit 29 network interface unit

Claims (5)

A multipoint video conference system in which a plurality of videophone / conference terminals are connected to a communication network,
Connect multiple video conference terminals to two different video conference terminals,
A communication path between the plurality of video conference terminals is connected in a ring shape by connection control information for performing connection control between the video conference terminals,
A multipoint video conference system, wherein a multipoint video conference is controlled by conference control information for designating a video conference terminal for transmitting video and specifying a configuration of a display screen thereof. The multipoint video conference system according to claim 1,
At least one video conference terminal of the plurality of video conference terminals is:
Connection control information setting means for setting connection control information for performing connection control between the video conference terminals so as to connect the plurality of video conference terminals in a ring shape;
Conference control information setting means for setting conference control information for controlling a multipoint video conference for designating a video conference terminal for transmitting video and designating the configuration of the display screen;
Control data transmission means for transmitting connection control information set by the connection control information setting means and conference control information set by the conference control information setting means as control data;
Video / audio data transmitting means for setting video / audio data based on the conference information and transmitting the video / audio data to the video conference terminal set in the connection control information;
A multipoint video conference system characterized by comprising: The multipoint video conference system according to claim 1 or 2,
Each of the video conference terminals
Connection control means for controlling connection and disconnection with a specific video conference terminal set in the connection control information in the received control data;
Video data control means for recognizing the configuration of the video conference terminal that transmits the video specified in the conference control information in the control data and the display screen thereof and configuring the display screen of the terminal itself;
A multipoint video conference system, comprising: control data relay means for relaying control data received so that the control data circulates on a loop constituted by the plurality of video conference terminals. The multipoint video conference system according to any one of claims 1 to 3,
Each of the video conference terminals updates video / audio data of the video / audio data sent from the other video conference terminals to new video / audio data and sends the video / audio data to the other video conference terminals. A multipoint video conference system characterized by having voice data updating means. The multipoint video conference system according to any one of claims 1 to 4,
The multipoint videophone conference system, wherein the communication network is an IP network.

Images