JP3390033B2 - Packet communication method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a packet communication system for packetizing and communicating information, and more particularly to a video communication system for packetizing video information and communicating it via a packet communication network.

[0002]

2. Description of the Related Art A conventional video communication system uses a circuit switching network such as ISDN as a communication network. When a circuit switched network is used, the communication band is guaranteed and the communication throughput between terminals becomes constant. Therefore, the video encoding device that compresses and encodes the video information input by the video input device such as a camera, that is, the codec may output the encoded video information at a constant speed according to the communication band of the communication network.

On the other hand, in recent years, information networks of offices have been developed, and a large number of personal computers and workstations connected to each other by LAN have been introduced. Therefore, using these personal computers and workstations,
There is an increasing need to build a video communication system such as a V telephone. However, the LAN is a communication network that performs packet switching unlike a circuit switching network such as ISDN.
The communication throughput between terminals fluctuates due to factors such as the load on the communication network. Therefore, the conventional codec that outputs encoded video information at a constant speed cannot be used as it is.

One solution to this problem is "Materials of the Institute of Electronics, Information and Communication Engineers, OS90-46, pp.31-"
36, 1990, "A Study of Personal Multimedia Communication Computer (PMCC)". This is a method in which, during video coding, a part having a large influence on the video quality and a part having a small influence on the video quality are layered and coded, and are transmitted as separate packets having different priorities. However, with this method, only two-step control is possible, and the flexibility of dealing with fluctuations in communication throughput is not sufficient. Therefore, a codec for packet switching capable of controlling the output speed of encoded video information from the outside is required.

[0005]

When a codec for packet switching is realized, if a dynamic change in communication throughput between terminals can be measured, the coding rate of the codec is controlled by following the throughput. Video communication on the packet switching network becomes possible. The problem here is how to measure the communication throughput between terminals.

An object of the present invention is to provide means for measuring a dynamic change in communication throughput between terminals.

Another object of the present invention is to provide means for enabling video communication on a communication network such as a LAN whose communication throughput dynamically changes.

Another object of the present invention is to provide means for accumulating coded video information from a coding device whose coding speed dynamically changes and for reproducing and displaying it offline. It is in.

[0009]

SUMMARY OF THE INVENTION An object of the present invention is to make it possible to control the video coding speed of a coding device from the outside, and indicate the coding time in each communication packet in coded video information communication between video terminals. Information is added, the packet reception time is recorded on the receiving side, and the receiving side evaluates the magnitude relation between the coding speed and the communication throughput by comparing the difference between the coding time within each packet and the packet receiving time. To be achieved. Further, by notifying the transmission side terminal of this evaluation result and controlling the video encoding speed of the encoding device in accordance with the evaluation result at the transmission side terminal, efficient video communication becomes possible.

Another object of the present invention is to store the video coding time of the coding device together with the coded data in the storage device, and to reproduce the difference of the stored time information when it is reproduced offline. restore the video encoded data accumulated based on
Is achieved by controlling the time at which input to No. apparatus.

[0011]

According to the present invention, when the difference between the packet generation interval at the transmission terminal and the packet reception interval at the reception terminal is larger than the threshold value, the packet generation interval at the transmission terminal, that is, the coding rate is higher than the communication throughput. The receiving terminal notifies the transmitting terminal of this fact, and the transmitting terminal side controls so as to reduce the coding rate. On the contrary, if the difference between the packet generation interval at the transmitting terminal and the packet receiving interval at the receiving terminal is smaller than the threshold value, there is a possibility that the coding rate at the transmitting terminal is still larger than the communication throughput. Then, the receiving terminal notifies the transmitting terminal of this fact, and the transmitting terminal side controls so as to increase the coding rate.

[0012]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the drawings. FIG. 1 is an overall configuration diagram of a video communication system according to an embodiment of the present invention. The video communication system is a terminal group 100 for performing video communication.
(100-1, 100-2, ...) And the packet communication network 120 connecting them to each other. Each terminal 1
Reference numeral 00 denotes a video information input means 101, a coding means 102 for compressing and coding the video information, a time information section 103 for obtaining time information given to a code, and a coding speed control for controlling a coding speed. Means 104, a communication control unit 105 for transmitting and receiving the encoded video information to the network.

Further, on the side receiving the video information, the coding speed adjusting means 106 for adjusting the coding speed, the time information section 103, and the decoding means 10 for decoding the video information.
7. Video information display means 108 and time information table 80
It has 0.

FIG. 2 shows a CPU of the video communication terminal 100 shown in FIG.
It is an example configured by using a memory and a program for video communication processing. The terminal 100 includes a CPU 110 for reading out a program for video communication processing from the memory 111 and controlling the operation of the video terminal. Further, it is provided with a memory 111 for temporarily storing this program and compression-encoded video information. Further, a storage device 112 for storing compression-encoded video information as a file and a bus 10 for connecting the respective parts to each other.
9 is equipped.

FIG. 3 shows the CPU 110 of the video communication terminal 100.
Shows the configuration of the program that operates in. The program that operates on the CPU 110 includes an operating system 201 that controls the operation of the entire program and a video communication process 202 that performs the main video communication process.

FIG. 4 shows the internal structure of the encoding device 102. The encoding device 102 is a coder unit 301 that performs a compression encoding process of the video information from the video input device 101, and temporarily stores the compression encoded video information in the encoding device in order to deliver it to the CPU 110. FIFO 30
2. From the encoding device 102 to the CPU 110, a FIFO (first-in first-out buffer) 302
The length register 303 for transmitting the length of the encoded information in the inside, the encoding device 102 to the CPU 110 completes the compression encoding process of one video frame, and the encoded information is FI.
An interrupt flag 304 for notifying that the data is stored in the FO 302, a command register 305 for setting various commands from the CPU 110 to the encoding device 102,
It comprises a signal processor 306 for controlling the operation of the encoding device 102. Command register 305
The commands issued from the CPU 110 to the encoding device 102 via the command include a START command and a STOP command.

FIG. 5 shows the internal structure of the decoding device 107. In the decryption device 107, the CPU 110 has the decryption device 10
FIFO 401, CP for writing coded information in 7
U110 is a length register 402 for transmitting the length of the encoded information in the FIFO 401 to the decoding device 107,
The decoder unit 403 that performs the decoding process of the encoded information in the FIFO and the decoding device 107 instructs the CPU 110 to complete the decoding process of one video frame, and the encoded information is transferred to the FIFO.
An interrupt flag 404 for notifying that writing is possible to 401, and a command register 4 for setting various commands from the CPU 110 to the decoding device 107
05, a signal processor 406 for controlling the operation of the decoding device 107. The commands issued from the CPU 110 to the decryption device 105 via the command register 405 include the START command and the STOP command.
There is a command.

Next, a video communication system between the video communication terminals 100 via the communication network 120 will be described. In the following, "-1" is assigned to each constituent device of the transmission side terminal and "-2" is assigned to each constituent device of the reception side terminal to distinguish them.

FIG. 6 shows a communication frame 60 transmitted and received between the transmitting side video terminal 100-1 and the receiving side video terminal 100-2.
The structure of 0 is shown. The communication frame 600 includes the communication network 120.
It is composed of a communication header 601 for performing communication above, a video communication header 602, and video communication data 603. Note that there may be no data portion depending on the type of frame as described later. Regarding the communication header 601,
It is a normal communication protocol header such as TCP / IP and will not be described in detail here. The video communication header 602 has a length of 1 octet, and contains a command exchanged between video terminals. The commands include a video communication start command, a video communication end command, a video data command, a coding speed reduction command, and a coding speed acceleration command. In the case of a video data command, there is a data section 603. The data section 603 is composed of a 2-octet length section 604, a 2-octet time section 605, and a compression encoded data section 606. The length section 604 stores the number of octets of the compression encoded data. In the time section 605, the generation time of the compression encoded video data obtained from the time information section 103 in the transmission video terminal 100-1 is entered.

FIG. 7 shows the CPU 110 of the video terminal 100-1.
2 is a processing flow on the transmission side of the video communication process 202 that operates according to the above. First, the video communication process 20 of the sending video terminal
2-1 is a command register 3 when starting video communication.
The START command is written in the command line No. 05 (711), and the video communication process 20 of the receiving-side video terminal 100-2 is executed.
A video communication start command frame is transmitted to 2-2 (712). Next, the presence / absence of an interrupt is confirmed from the encoding device 102 (713), and if there is no interruption, the encoding speed change command frame reception process is performed (714). If the command is received, the encoding speed is changed by changing the FIFO read waiting time according to the content of the command (71
5). If there is a stop request from the terminal user (7
16), returns to step 713, and if there is no, writes a stop command to the command register 305 (71
7) Then, a video communication end command frame is transmitted to the video terminal on the receiving side (718), and a series of processing is ended.

If there is an interrupt from the encoder 102 in step 713, after waiting for the FIFO read waiting time, the FIFO is read at a preset default speed.
The compression-coded video information 606 and the video information length 604 are read from the O 302 and the length register 303 (720). To read at the default speed,
A method of starting the video communication process 202-1 from the operating system 201-1 of the video terminal at a constant cycle is conceivable. At the same time, a system call for timer management is issued to the operating system 201-1 to obtain the current time 605 from the time information section 103 (7
21). After that, the video information command frame 600 is created from the time information 605, the length information 604, and the compression-encoded video information 606, and the receiving side video terminal 100
-2 (722). In this way, the video communication process 202-1 on the transmission side repeats the processing from the reading of the compression-encoded video information to the transmission of the video information command frame in a constant cycle.

On the other hand, the CPU of the receiving side video terminal 100-2
As shown in FIG. 8, when the video communication process 202-2 operating in 110 receives a video communication start command frame (731), it writes a start command in the command register 405 (732). Further, when the video information command frame 600 is received (733), the length information 604 and the compression-encoded video information 606 in the video information command frame are transferred to the length register 402 and the FIFO 4.
01 is written (736), a timer management system call is issued to the operating system 201-2 in the own terminal to obtain the current time (737), and the time information 605 in the received video information command frame is associated. It is made into a table and stored in the time information table 800 (738). In the video communication process 202-2 of the video terminal 102-2 on the receiving side, the above processing is repeated every time the video information command frame is received.

Upon receiving a certain number of video information command frames 600, the video communication process 202-2 next uses the time information in the time information table 800 to perform a coding speed adjustment process (739). When it is determined by the coding rate adjustment processing that the coding rate is changed (740),
Video communication process 202- of the receiving-side video terminal 100-2
2 transmits the coding speed change command frame to the video communication process 202-1 of the sending video terminal 100-1 (741). If there is an interrupt from the decoding device 107 (742), the process returns to step 733. If the video information command frame has not been received, it is next checked whether or not the video communication end command frame has been received (734), and if it has been received, the command register 40
A stop command is written in 5 (735), and a series of processing is ended.

Next, the operation of the encoding device 102 will be described.
FIG. 9 shows a processing flow of the signal processor 306 that controls the overall operation of the encoding device 102. First, the signal processor 306 starts the START from the CPU 110.
Wait for a command (311). Specifically, it searches for the START command to be written in the command register 305. When the START command is issued, the coder 30
A video encoding instruction is issued to 1 (312). Then the CPU
Is a STOP command issued from 110 or coder 30?
It waits for issuance of the compression-encoding process end notification of 1 to 1 video frame (313, 314). When the STOP command is issued from the CPU 110 (313), the operation end instruction is given to the coder 301 (315), and the process flow returns to the beginning.

When the compression coding end notification of one video frame is received from the coder 301 (314), the length of the compression coding information stored in the FIFO 302 is received from the coder 301 and set in the length register 303. After that (316), the interrupt flag 304 is set to interrupt the CPU 110 (317).
When the CPU 110 receives the interrupt, the length register 3
03 and FIFO 302 to obtain the compression-coded video information and its length. After the signal processor 306 interrupts the CPU 110, the FIFO3
When 02 becomes empty, it is monitored (318), and when it becomes empty, the processing from the video coding instruction issue to the coder 301 (312) is repeated. In this way, the encoding device 102 operates in the CPU 110 within a range equal to or lower than the video encoding maximum speed.
The video encoding speed is controlled by the reading speed of the information in the FIFO 302.

Next, the operation of the decoding device 107 will be described.
FIG. 10 shows a processing flow of the signal processor 406 that controls the overall operation of the decoding device 107. First, the signal processor 406 starts the STAR from the CPU 110.
Wait for the T command (411). Specifically, it searches for the START command to be written in the command register 405. When the START command is issued, CPU1
Wait until the STOP command is issued from 10 or the length is set in the length register 402 (41
2, 413). When the STOP command is issued from the CPU 110 (412), the initial state is restored.

After the START command is issued, the CPU 110 writes the compression-coded video information in the FIFO 401, and then sets the length of the information written in the FIFO 401 in the length register 402. When the length is set in the length register (413), the signal processor 401 notifies the decoder 403 of the length,
Issue a decryption instruction. The decoder 403 is the FIFO 40
The compression-coded video information in 1 is decoded, and the decoded video information is sent to the video display device 108, and then the signal processor 406 is notified of the end (414). When the signal processor 406 receives the end notification from the decoder 403 (415), it sets the interrupt flag 404 and interrupts the CPU 110 (416), and then the CP
It waits for the next input of compression coded video information from U110. C
When receiving an interrupt, the PU 110 inputs the next compression-coded video information. As described above, the present decryption device is
The compression-coded video information received from 10 is sequentially decoded.

FIG. 11 is a communication sequence chart between the transmitting side video terminal 100-1 and the receiving side video terminal 100-2 corresponding to the processing flows described in FIGS. First, when the video communication process 202-1 of the transmitting side video terminal 100-1 starts the video communication, it transmits a video communication start command frame to the video communication process 202-2 of the receiving side video terminal 100-2 and ( 90
1), STAR for the encoding device 102-1 in the own terminal
A T command is issued (902). Receiving side video terminal 10
When the video communication process 202-2 of 0-2 receives the video communication start command frame, the decoding device 1 in its own terminal 1
Issue a START command to 07-2 (90
3).

The video communication process 202-1 on the transmitting side issues the START command to the encoding device 102-1 and then the encoding device 102 at a predetermined default speed.
The compression-encoded video information and the length of the video information are read from -1 (904), and at the same time, the current time is obtained (90).
5). After that, a video information command frame is created from these time information, length information, and compression-encoded video information, and transmitted to the receiving-side video terminal 100-2 (906).
The video communication process 202-1 on the transmission side repeats the processing from the compression-coded video information read to the video information command frame transmission at a constant cycle.

Upon receiving the video information command frame, the video communication process 202-2 of the receiving-side video terminal 102-2 sends the length information and the compression-encoded video information in the video information command frame to the decoding device 107-2. At the same time as writing (907), a system call for timer management is issued to the operating system 201-2 in the own terminal to obtain the current time (908), and the time information table is associated with the time information in the received video information command frame. Create 800 (909). When a certain number of video information command frames are received, the time information in the time information table 800 is used to perform the coding speed adjustment processing (91).
0).

FIG. 12 shows the structure of the time information table 800. The time information table is composed of in-packet time information 811 and reception time information 812 for each received packet 810. In the example of the table, the packet number 10 is the oldest and the packet number 22 is the latest packet.

FIG. 13 shows a detailed processing flow of the coding speed adjustment processing in step 739 of FIG. In the coding rate adjustment processing 739, the difference between the latest in-packet time information 811 in the table 800 and the oldest (oldest) in-packet time information 811 is obtained (1020), which is Δ.
t. Next, the latest packet reception time in the table
812 and obtains the difference between the packet reception time 812 of the re-former (1021), which is referred to as [Delta] T. Next, the difference between ΔT and Δt is obtained (1022) and is set as Δ. Then Δ and Δ
The ratio with t is calculated (1023). Δt is the table 80
The latest in-packet time information 811 in 0 and the old packet
Since it is the difference with the internal time information 811, it is divided by this Δt.
Means to take the average value of the difference Δ between ΔT and Δt.
I'm tasting. In the example of FIG. 12, Δt = 750−150 =
600, ΔT = 1780-1002 = 778, Δ = ΔT
-Δt = 178 and Δ / Δt = 0.297. If this ratio is larger than a predetermined threshold value, it is determined that the coding rate is larger than the throughput of the communication network (102).
4). On the contrary, when it is small, it is determined that the throughput of the communication network can sufficiently follow the coding rate (1025).
The threshold value is obtained in advance by simulation or the like.

Returning to FIG. 11, the coding speed adjustment processing 91
When it is determined that the coding rate is higher than the throughput of the communication network by 0, the video communication process 202-2 of the receiving side video terminal 100-2 sends the coding rate slowing command frame to the transmitting side video terminal 100-1. It is transmitted to the video communication process 202-1 (911). The transmitting side video terminal 10 that has received the encoding speed reduction command frame
The video communication process 202-1 of 0-1 is the encoding device 1
The reading cycle from 02-1 is slowed down (912).

On the other hand, when it is determined that the throughput of the communication network can sufficiently follow the coding rate by the coding rate adjustment processing, the video communication process 202-2 of the receiving side video terminal 100-2 determines the coding rate. Video communication process 202 of transmitting side video terminal 100-1
To -1 (913). The video terminal 100-1 on the transmitting side which has received the command frame for increasing the encoding speed
Of the video communication process 202-1 of the encoding device 102-
The read cycle from 1 is accelerated (914). In addition,
Values for the speed-up and speed-down pitches are set in advance.

According to the present embodiment, it is possible to perform video communication on a communication network in which traffic control such as LAN is difficult and communication throughput dynamically changes. Videophone or TV using a connected workstation or personal computer
There is an effect that a multimedia application that requires video communication such as a meeting can be realized.

Next, as another embodiment of the present invention, a case where compression-encoded video information is stored in the storage device 112 of the receiving side video communication terminal 100-2 and is reproduced and displayed offline. An example is shown in FIG.

Also in this embodiment, the processing of the video communication process 202-1 in the transmitting side video communication terminal 100-1 is the same as that of the above-mentioned embodiment. Receiving side video communication terminal 10
The process of the video communication process 202-2 in 0-2 is divided into two processes, a process at the time of receiving the compression coded video information and an off-line reproduction process.

First, when the compression-encoded video information is received, the time information, the length information, and the video data in the received video data command frame are sequentially stored in the storage device 112-2. Next, FIG. 14 shows a processing flow of the offline reproduction processing. First, the time information, the length information, and the video data for the length are read from the storage device 112-2 (1431). Next, the time information is saved in the memory, and the length information and video data are decoded by the decoding device 1
The data is written in 07-1 (1432). After that, the storage device 1
The next time information is read from 12-2 and the difference from the time information previously saved in the read memory is obtained (14
33). Next, weighting is performed for a time corresponding to the difference of the time information (1434). After the wait, the length information and the video data are read from the storage device 112-2, and the decoding device 10
Write to 7-2. The above processing is repeated until there is no video data in the storage device 112-2 (1435). According to this method, even when the coding speed in the video communication terminal on the transmitting side dynamically changes, the coded video information can be accumulated and reproduced and displayed offline at an appropriate speed.

[0039]

According to the present invention, since the video coding speed of the coding device can be dynamically changed according to the throughput of the communication network, the communication throughput of the LAN or the like is dynamically changed. This has the effect of enabling video communication.

[Brief description of drawings]

FIG. 1 is an overall configuration diagram of a video communication system according to an embodiment of the present invention.

FIG. 2 is a diagram showing a detailed configuration of the video communication terminal shown in FIG.

FIG. 3 is a diagram showing a software configuration of a video communication terminal.

4 is a diagram showing the configuration of the encoding device in FIG.

5 is a diagram showing the configuration of the decoding device in FIG.

FIG. 6 is a diagram showing a configuration of a communication frame.

FIG. 7 is a diagram showing a transmission side processing flow of the video communication terminal.

FIG. 8 is a diagram showing a reception side processing flow of the video communication terminal.

FIG. 9 is a diagram showing a processing flow in the encoding device.

FIG. 10 is a diagram showing a processing flow in the decoding device.

FIG. 11 is a diagram showing a communication sequence chart.

FIG. 12 is a diagram showing the configuration of a time information table.

FIG. 13 is a diagram showing a processing flow of encoding speed adjustment processing.

FIG. 14 is a diagram showing a processing flow of offline reproduction processing.

[Explanation of symbols]

100 ... Video communication terminal, 101 ... Video input device, 102
... Encoding device, 104 ... Encoding speed control means, 106 ...
Coding speed adjusting means, 107 ... Decoding device, 108 ... Video display device, 110 ... CPU, 112 ... Storage device, 11
1 ... memory, 120 ... communication network,

─────────────────────────────────────────────────── ─── Continuation of the front page (72) Toru Hoshi Inventor Toru Hoshi 1099, Ozenji, Aso-ku, Kawasaki-shi, Kanagawa, Ltd. System Development Research Laboratory, Hitachi, Ltd. (58) Fields investigated (Int.Cl. ⁷ , DB name) H04L 12 / 56 200

Claims (5)

(57) [Claims] 1. A coding terminal comprising an input device, a coding processing unit for coding input information, and a communication processing unit for packetizing and transmitting the coded information, and the packetized information. A decoding terminal including a communication processing section for receiving, a decoding processing section for decoding the coded information, and a display device for displaying the decoded information, the coding terminal and the decoding A packet communication method in a communication system configured by a packet communication network for connecting terminals to each other and packet-communicating the encoded information, wherein each time of packetizing the information in the encoding terminal is Obtained, set the time information in the packet to be transmitted, at the decoding terminal that received the packet, obtain the time at the time of receiving the packet, and receive the two packets The difference Δt between the times set in step 1 and the time ΔT at the time when the packet is received is obtained, the difference Δ between these two differences is obtained, and the difference Δ between these differences is averaged over a plurality of received packets. A packet communication method, wherein a value is obtained, and the average value is used to control an encoding speed in an encoding processing unit of the encoding terminal. 2. If the average value is larger than a predetermined value, the decoding terminal notifies the coding terminal to the coding terminal as coding rate control data, and the coding terminal receiving the control data sets the coding rate control data. , packet communication method according to claim 1, wherein the slowing encoding rate of the encoding processing unit by the coding rate control means. 3. If the average value is smaller than a predetermined value, that effect is transmitted from the decoding terminal to the coding terminal as coding rate control data, and the coding terminal receiving the control data , packet communication method according to claim 1, characterized in that the speed of the encoding rate of the encoding processing unit by the coding rate control means. 4. The decoding terminal comprises a data storage device,
In the one that stores the encoded information from the encoding terminal in the data storage device and then decodes and reproduces the encoded information that has been stored offline, the Interval at which time information is stored in the data storage device in association with the encoding information in the packet, and the associated encoding information is input to the decoding processing unit based on the time information during reproduction. The packet communication method according to claim 1 , wherein the packet communication method is controlled. 5. An encoding terminal comprising an input device, an encoding processing unit for encoding input information, and a communication processing unit for packetizing and transmitting the encoded information, and the packetized information. A decoding terminal including a communication processing section for receiving, a decoding processing section for decoding encoded information, and a display device for displaying the decoded information, the encoding terminal and the decoding terminal In a communication system including a packet communication network that interconnects each other to perform packet communication of the encoded information, the encoding processing unit is capable of controlling the amount of information to be encoded per unit time. The decoding terminal comprises a speed control means and means for determining each time when the information is packetized and setting the time information in a packet to be transmitted, wherein the decoding terminal obtains the time when the packet is received, The difference between the time set in the two received packets and the time difference at the time of receiving the packet is obtained as an average value for a plurality of received packets, and the average value is compared with a predetermined value, A packet communication system comprising: a coding rate adjusting means for transmitting a result as control data to a coding rate controlling means of the coding terminal.