JP3426144B2 - Multimedia communication equipment - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to inter-application communication executed by an information processing device such as a personal computer or a portable information terminal operating on a network such as a LAN, and more particularly to a packet-based protocol (TCP).
The present invention relates to a multimedia communication device that performs packet communication using (UDP) / IP).

[0002]

2. Description of the Related Art Conventionally, when image, voice or simultaneous communication of these is carried out using a packet communication line, the international standard I
TU-TH. In the case of performing communication conforming to H.323, the communication protocol RTP (A Transport Protocol for Real-Time) is used.
Applications) and communication using UDP.
Hereinafter, the case of performing image transmission will be described with reference to FIG. 7.

First, the transmission terminal 3 takes in an image to be transmitted from the image input section 30. An image encoding unit 31 converts the captured image signal into information that has been compressed / encoded appropriately for communication. This converted information is sent to the transmitter 3
Passed to 2. In the transmitting unit 32, first, in the RTP packet assembling unit 321, in order to transmit the image information to the receiving terminal 4 using the RTP protocol, information such as a header necessary for the image information is added to generate the RTP packet. . The UDP packet assembling unit 322 adds information such as a header to the generated RTP packet to generate a UDP packet. The generated UDP packet is sent from the packet transmission unit 323 to the packet communication line 5.

In the receiving terminal 4, the packet arriving from the packet communication line 5 is received by the receiving section 43. In the receiving unit, first, the packet receiving unit 433 receives a packet from the communication line. Next, the received UDP packet is passed to the UDP packet decomposing unit 432, and the RTP packet is extracted from the UDP packet. The RTP packet disassembly unit 431 extracts only the image information from the extracted RTP packet. The image decoding unit 42 decompresses this information from the encoding / compression, and the image output unit 41 extracts the image signal. As described above, the image information to be transmitted is placed on the RTP packet, and the RTP packet is UD.
The packet is placed on the P packet and transmitted on the packet communication line.

Here, UDP (User Datagram Protoco
l) is a protocol that emphasizes high-speed transmission rather than reliability. FIG. 5 shows a transmission procedure when transmitting 1 to n UDP packets. As shown in FIG. 5, in UDP, only a packet is sent to a partner terminal, and other processing such as retransmission is not performed, so that high throughput transmission can be performed. However, even if the transmitted packet is incorrect, the packet is not retransmitted. As described above, UDP has an advantage that high-speed data transmission can be performed although it is not reliable, and is suitable for data transmission that requires real-time characteristics such as moving image data. UDP is used for transmitting voice coded data.

In order to improve this low reliability, in Japanese Patent Laid-Open No. 10-56479, a packet serial number is given in the user data area of a UDP packet,
When the receiving terminal side detects the discontinuity of the serial numbers, the method of notifying the transmitting terminal of the missing information and requesting the retransmission of the transmitting terminal side is adopted. In this method, the upper protocol of the UDP protocol needs to implement and implement the original protocol described in the publication.

[0007]

On the packet communication line, packet loss occurs due to congestion in the line and occurrence of transmission error in the wireless section of the line. In the conventional technology, the image and sound are converted into UDP packets,
Send this to the other party. As mentioned above, UDP is a non-guaranteed communication protocol, and is a so-called “keep sending” communication protocol. Even in the environment where packets are lost, it is not possible to realize even if the user requests "I want to transmit an image accurately because the transmission may take some time". Further, the implementation method in the above publication is also
Usually ITU-T Recommendation H.264. When performing image or voice communication conforming to H.323, the RTP protocol is used as the upper protocol of the UDP protocol, and if the protocol shown in the publication is adopted instead of the RTP protocol, mutual communication between the respective terminals is possible. Connection is no longer guaranteed.

[0008]

In order to solve the above-mentioned problems, according to the present invention, when a packet loss occurs, the sender side uses the reliable TCP protocol in parallel with the UDP protocol transmission. Transfer and send image or sound. The receiving side receives either UDP or TCP. TCP (Transmission Control Protocol)
l) is a protocol that enables highly reliable communication. The receiving side sends back an ACK to the sending side every time a packet is received, and the sending side waits for the ACK to be sent before the next packet. To send. If there is an error in the received packet on the receiving side, NACK is sent back and retransmission is performed. FIG. 6 shows a transmission procedure when transmitting 1 to n TCP packets. As shown in FIG. 6, data transmission is guaranteed in TCP, but the procedure required for data transmission is more complicated than that in UDP, which makes it relatively unsuitable for data transmission that requires real-time processing. Further, the receiving side has a function of monitoring the packet loss of the UDP protocol so that the missing information can be transmitted to the transmitting side. Further, by making it possible to receive image or audio information by either the UDP protocol or the TCP protocol, it becomes possible for the transmitting side to simultaneously select and receive information transmitted by the two protocols.

According to the invention of claim 1, multimedia information such as image information or audio information is transmitted through a packet communication line.
In multi-media communication apparatus for performing packet communications with each other Te, as a function of transmitting to the counterpart terminal, to create a means for creating a RTP packet, means for creating a UDP packet from the RTP packet, the TCP packet from the RTP packet And a means for sending a UDP packet or a TCP packet to the communication line, and a means for receiving packet loss information coming from the partner terminal. As a function of receiving from the partner terminal, the packet communication line sends data from the partner terminal. Received UDP packet or TCP
Means for receiving packets and RTP from UDP packets
Means for extracting packets and RTP from TCP packets
Means for extracting packets, means for extracting multimedia information from the extracted RTP packets, UDP
Means for counting the number of dropped packets and counted UDP
And means for transmitting to the other terminal of the number of missing packets as a packet loss information, the packet deletion arriving from the remote terminal
Depending on the drop information, multimedia information by UDP packet
TCP packet transmission is performed in parallel with the information transmission.
It was done like this.

[0010]

[0011]

A second aspect of the present invention relates to a computer-readable recording medium having a program for causing a computer to function as the multimedia communication device according to the first aspect.

[0013]

According to the multimedia communication device of the present invention, the receiving side can receive the same data. Therefore, when it is desired to accurately receive image or audio information, the TCP protocol is used. If the received image or audio information is received and packet loss occurs,
The retransmitting unit enables the receiving terminal side to reliably receive the image or audio information. In addition, ITU-TH.
For transmission of image or audio information to a terminal compliant with H.323, since the transmission side transmits image or audio information by both UDP / TCP, it suffices to receive the information carried by the UDP protocol. Can be secured.

The configuration of the embodiment of the present invention will be described. Figure 1
FIG. 3 is a diagram showing a communication connection form according to an embodiment of the present invention.
FIG. 2 is a block diagram of an embodiment of the transmitting unit in the transmitting terminal of the present invention. FIG. 3 is a block diagram of an embodiment of the receiving unit in the receiving terminal of the present invention. The embodiment of the present invention shows a case where an image input from the transmission terminal 1 is transmitted to the reception terminal 2 via the packet communication line 5. The operation of the embodiment of the present invention will be described below.

First, the image to be transmitted is fetched from the image input section 10. The captured image signal is converted by the image encoding unit 11 into information that has been appropriately compressed and encoded for communication. The converted information is passed to the transmitter 12. In the transmitting unit 12, first, in the RTP packet assembling unit 121, information such as a header necessary for the image information is added in order to transmit the image information to the receiving terminal side by using the RTP protocol, and the RTP packet is created. To generate. The generated RTP packet is processed by the UDP packet assembling unit 122.
A UDP packet is generated by adding information such as a header.
The generated UDP packet is sent from the packet transmitting unit 124 to the packet communication line.

In the receiving terminal 2, the packet arriving from the packet communication line 5 is received by the receiving unit 22. Receiver 2
In 2, the packet receiving unit 221 first receives a packet from the communication line. Next, the UDP packet decomposing unit 22
The received UDP packet is passed to 2, and the RTP packet is extracted from the UDP packet. The RTP packet decomposition unit 224 extracts only the image information from the extracted RTP packet. Since the image information is transmitted by the UDP packet at the start of communication, the SW 228 is set to b.
It has fallen to the side. The extracted information is the image decoding unit 21.
At, the image signal is extracted from the image output unit 20 by performing the decoding from the encoding / compression.

However, when the packet transmitted from the transmitting terminal cannot be received on the receiving terminal side, the UDP packet disassembling unit 222 detects it, and the counter 227 counts the number of packets that cannot be received. Here, as a method of detecting that the packet could not be received, the packet serial number is added in the header of the UDP packet, and this number was confirmed every time the packet was received, and the discontinuity of the number could be detected. It is possible to detect that there are packets that could not be received at the time. The transmission control unit 226 constantly monitors this counter and notifies the missing information transmitting unit 225 of the value of the counter as the number of missing packets at a certain fixed interval, and the missing information transmitting unit sends the missing information to the transmitting terminal side at an appropriate timing. Send to. On the other hand, in the transmitting terminal, the missing information received from the receiving terminal is received by the missing information receiving unit 125.
When it is received at, the reception content is notified to the transmission control unit 126. In this way, both terminals can know the packet loss. The means for notifying the missing information can be realized by using the RTP control protocol (RTCP).

Therefore, at the time of starting communication, a threshold value for the number of packet loss is determined between the terminals in advance, and when the packet loss at the receiving terminal side exceeds the threshold value, S
By switching W228 from the b side to the a side, it is possible to receive the RTP packet that has arrived on the TCP packet from the transmission terminal. Thereafter, the packet received by the packet receiving unit 221 is transferred to the TCP packet decomposing unit 22.
3, the header and the like are removed, and then the extracted RTP packet is delivered to the RTP packet decomposing unit 224 via the SW 228. On the other hand, on the transmission terminal side, when the transmission control unit 126 recognizes that the packet loss notified from the reception terminal has exceeded the threshold value, the SW 127 is closed and one RTP packet is divided into a UDP packet and a TCP packet.
It is sent to the receiving terminal side using the type of packet.

As a result, U is set between the transmitting terminal and the receiving terminal.
The image information transmitted using the DP protocol is transferred to the T
It becomes possible to perform transmission by switching to the CP protocol, and reliable transmission is performed by the retransmission function of the TCP protocol. When the receiving terminal side exceeds the threshold value for packet loss, even if packet loss etc. occurs after this point, if you want to avoid the drop in throughput due to retransmission and continue communication, set SW to b side. It is also possible to continue the transmission by the UDP protocol as it is. Also,
FIG. 4 shows a connection diagram in the case where the receiving terminal side is the conventional receiving terminal 4 that cannot switch the UDP / TCP packet reception shown in the present invention. In the case of this connection, even if packet loss occurs, the packet loss information is not notified from the receiving terminal 4 to the transmitting terminal 1, so that it is possible to perform transmission using communication according to the UDP protocol.

[0020]

In the communication using the device of the present invention, it is possible to transmit and receive high-definition images or sounds.
In addition, since the image or audio information is also transmitted as a UDP packet, even if the other party is an international standard compliant terminal, communication can be performed without any problem.

[Brief description of drawings]

FIG. 1 is a block diagram showing a communication connection form according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an embodiment of a transmission unit of the present invention.

FIG. 3 is a block diagram showing an embodiment of a receiving unit of the present invention.

FIG. 4 is a block diagram showing a communication connection form with a conventional receiving terminal according to another embodiment of the present invention.

FIG. 5 is a sequence diagram showing communication between terminals using UDP.

FIG. 6 is a sequence diagram showing communication between terminals using TCP.

FIG. 7 is a block diagram showing conventional image communication using packets.

[Explanation of symbols]

1 ... Sending terminal, 2 ... Receiving terminal, 3 ... Conventional sending terminal, 4
... conventional receiving terminal, 5 ... packet communication line, 10 ... image input section, 11 ... image coding section, 12 ... transmission section, 20 ... image output section, 21 ... image decoding section, 22 ... receiving section, 30 ...
Image input unit, 31 ... Image coding unit, 32 ... Transmission unit, 41
... image output unit, 42 ... image decoding unit, 43 ... receiving unit, 1
21 ... RTP packet assembling section, 122 ... UDP packet assembling section, 123 ... TCP packet assembling section, 124 ... Packet transmitting section, 125 ... Missing information receiving section, 126 ... Transmission control section, 127 ... SW, 221 ... Packet receiving section, 222
... UDP packet decomposing unit, 223 ... TCP packet decomposing unit, 224 ... RTP packet decomposing unit, 225 ... Missing information transmitting unit, 226 ... Transmission control unit, 227 ... Counter, 22
8 ... SW, 321 ... RTP packet assembling section, 322 ... U
DP packet assembling unit, 323 ... Packet transmitting unit, 431
... RTP packet decomposing unit, 432 ... UDP packet decomposing unit, 433 ... Packet receiving unit.

Continuation of the front page (56) Reference JP-A-4-30251 (JP, A) JP-A-6-252896 (JP, A) JP-A-8-56221 (JP, A) JP-A-10-200897 (JP , A) Yoshihiro Ito, Masami Ishikura, Tohru Asami, Evaluation of a traffic generator / analyzer for real-time communication characteristics evaluation, IEICE Technical Report, Japan, The Institute of Electronics, Information and Communication Engineers, June 20, 1997. , IN97-47 Toshihiko Yamato, Knowing the Protocol and Making the Engineer's Way in a Flexible and Inflexible Way, The Electric Network Kenkaku Business (1st), Windows NT PRESS No. 3, Japan, Ltd. Technical Review, September 1, 1998, 184-189 (58) Fields investigated (Int.Cl. ⁷ , DB name) H04L 12/56 H04L 12/28 H04L 29/06

Claims (2)

(57) [Claims] 1. A multimedia communication device for performing mutual packet communication of multimedia information such as image information or audio information via a packet communication line to a counterpart terminal, and means for creating an RTP packet, It has means for creating a UDP packet from an RTP packet, means for creating a TCP packet from an RTP packet, means for sending a UDP packet or TCP packet to a communication line, and means for receiving packet loss information coming from a partner terminal. As a function of receiving from the partner terminal, a means for receiving a UDP packet or a TCP packet sent from the partner terminal from the packet communication line, a means for extracting an RTP packet from the UDP packet, and an RTP packet from the TCP packet. Means for extracting and said extracting And a means for extracting multimedia information from the counted RTP packet, a means for counting the number of missing UDP packets, and a means for transmitting the counted number of missing UDP packets to the other terminal as packet missing information. Due to the packet loss information that came more, U
A multimedia communication device, characterized in that transmission is performed by TCP packets in parallel with transmission of multimedia information by DP packets. 2. A computer-readable recording medium, which records a program for causing a computer to function as the multimedia communication device according to claim 1 .