JP2002208953A - Communication system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress any sound jump or the interruption of a video without increasing the buffer capacity of reception side equipment. SOLUTION: In this communication system, an encoder 3 transmits time agents A1, A2,..., Am-1, and Am having the transmission time of packets from first communication terminal equipment 1 as one of time information to second communication terminal equipment 2, and a decoder 4 which receives those time agents transmits time agents A1, A2,..., Am-1, and Am having the reception time of packets as one of time information to the encoder 3. Then, the encoder 3 detects a delay time DT based on the time information of the time agents, and sets a compression ratio R based on the detected delay time DT. Thus, it is possible to reduce the data quantity when the delay is large.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication system for transmitting audio and video data in packets.

[0002]

2. Description of the Related Art Conventionally, in a communication system using packets (sometimes called "cells"), a transmitting device transmits and receives original signals (audio signals and video signals) in an encoded state. An encoding / decoding technique in which a side device decodes a reproduced signal is used for transmission of audio and video.

In such a communication system, it is required that packets are received at the same interval as the transmission interval on the transmission side. However, since the transmission time of the transmission path is not always constant, the packet arrival interval fluctuates (jitter). ) May occur. If the jitter increases, the STC (System Time Clock) of the encoder on the transmitting side and the STC of the decoder on the receiving side may be shifted to affect the reproduction result. Therefore, for example, Japanese Patent Application Laid-Open No.
As described in Japanese Patent Application Laid-Open Publication No. H10-107, a technique of synchronizing with the STC of the encoder without causing a large shift in the STC of the decoder is proposed. In this technique, received data is temporarily stored in a buffer, and a predetermined transfer rate (specifically, time information included in the received data and STC of a decoder are used).
By reading from the buffer at a transfer rate set based on the result of comparison with the STC, the effect of jitter on the synchronization between the STC of the decoder and the STC of the encoder is suppressed. Further, by reading the received data after temporarily storing it in the buffer, the reproduced video and reproduced audio are not disturbed by a certain amount of jitter.

[0004]

However, if the arrival of a packet is delayed due to the occurrence of a large jitter larger than an assumed value, there is a possibility that a skipped sound is generated in a reproduced sound, or a discontinuity or a chip is generated in a reproduced image. . That is, if the arrival of a packet is delayed, encoded data necessary for reproduction may be lost even at the timing to be reproduced, and as a result, skipping of sound or loss of video may occur. There is.

In order to suppress such a problem, it is sufficient to increase the capacity of a buffer for storing received packets and to delay reading from the buffer.
Increasing the memory capacity in such a way increases the manufacturing cost of the receiving device and limits the expansion of the memory capacity, so if the packet delay is further increased, it will also affect the playback results. Will come out.

SUMMARY OF THE INVENTION The present invention has been made in view of such a problem, and an object of the present invention is to suppress skipping of sound and interruption of video without increasing the buffer capacity of a receiving device.

[0007]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, in the communication system of the present invention (claim 1), encoded data corresponding to an original signal is transmitted at a predetermined encoding rate. And a first communication terminal device (hereinafter, referred to as a first communication terminal device) that divides the encoded data into packets and sends the packets to a transmission path. When a packet arrives via this transmission path, the packet And a second communication terminal device (hereinafter, referred to as a second communication terminal device) that decodes the encoded data included in the data into a reproduction signal.

[0008] The second communication terminal generates delay state information based on the arrival timing of the packet, and transmits this to the first communication terminal. This delay state information is information for detecting a delay state of packet transmission on the transmission path. Although various forms of the delay time information are conceivable, for example, as described later, “the time difference between the transmission time of the packet from the first communication terminal device and the arrival time of the packet to the second communication terminal device (that is, , Transmission time), and “arrival frequency information indicating the frequency of arrival of packets to the second communication terminal device”.

[0009] The first communication terminal device changes the data amount of the coded data by setting the coding rate based on the delay state information transmitted from the second communication terminal device. The coding rate is, for audio signals, the sampling frequency and quantization number, and for video signals,
Since they vary depending on the spatial resolution, the number of video frames / second, the gradation, and the compression rate (more specifically, the quantization table value by DCT coding, etc.) in the case of performing the compression coding, these are changed to delay state information. , The coding rate can be set to a predetermined value.

[0010] That is, in the communication system of the present invention, the data amount of the encoded data is changed by setting the encoding rate according to the delay state information.
The larger the delay, the smaller the amount of encoded data is. Even if the transmission delay is large, if the data amount of the encoded data is reduced, it is possible to suppress a situation where the encoded data necessary for reproduction is missing. As a result, it is easy to obtain an uninterrupted playback signal at the time of decoding, and it is possible to suppress sound skipping when the original signal is an audio signal, and to suppress video skipping when the original signal is a video signal. Disruption can be suppressed.

As described above, as the delay state information, for example, "information capable of calculating a packet transmission time from the first communication terminal to the second communication terminal" can be considered.
That is, the delay state in the transmission path is detected by monitoring the change in the transmission time of the packet from the first communication terminal device to the second communication terminal device.

Since the generation time of the packet and the transmission time thereof are substantially the same, first, as described in claim 2, first, when the first communication terminal device generates each packet, it generates the packet. The time is recorded in each packet. Then, for the second communication terminal device, the “generation time” recorded in the packet and the “arrival time” of the packet are acquired, and “time difference information corresponding to the time difference between the generation time and the arrival time” is replaced with the delay state information. It is configured to generate as

[0013] In the communication system according to claim 2 configured as described above, based on "information capable of calculating a packet transmission time from the first communication terminal apparatus to the second communication terminal apparatus" as delay state information. Thus, the coding rate is set. In this case, whether the packet is transmitted regularly or irregularly,
The delay state on the transmission path can be grasped.

[0014] On the other hand, as in the communication system according to the third aspect, regardless of the presence or absence of encoded data to be transmitted as a packet, a first packet is generated at a predetermined time interval and transmitted to a transmission path. The communication terminal device may be configured such that the second communication terminal device generates arrival frequency information indicating a packet arrival frequency as delay state information.

In the communication system according to the third aspect of the present invention, even if there is no encoded data to be transmitted, packet transmission is performed periodically, so that the delay state can be always grasped. Thus, an appropriate coding rate can always be set for the delay state of the transmission path. Therefore, when the transmission of the encoded data by the packet is newly started, the effect of the communication system according to claim 1 can be promptly obtained. As the arrival frequency information, for example, information indicating “the number of packets arriving per unit time”, “the time interval of the arrival times of the packets” and the like can be considered.

The transmission of the delay state information from the second communication terminal to the first communication terminal can be performed by various means, but the communication from the second communication terminal to the first communication terminal is also transmitted in packets. In the configuration performed, claim 4
As described, the second communication terminal device may be configured to store and transmit the delay state information in the packet transmitted to the first communication terminal device.

According to this configuration, when data other than the delay state information is transmitted, the delay state information can be transmitted together, so that it is efficient, and the data is transmitted from the second communication terminal apparatus to the first communication terminal apparatus. An increase in the total data amount can be suppressed.

[0018]

An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is an explanatory diagram illustrating a schematic configuration of a communication system as one embodiment. This communication system includes a first communication terminal device 1 and a second communication terminal device 2.

The first communication terminal device 1 is provided with an encoder 3 for compressing and encoding an audio signal and a video signal (hereinafter referred to as an audio / video signal) as original signals at a predetermined compression rate R to generate audio / video data. Have been. On the other hand, the second communication terminal device 2 includes a decoder 4 for decoding a reproduction signal from audio / video data. The compression ratio R in the encoder 3 is variable and is set in an encoding process described later. The encoder 3 and the decoder 4 include a CPU, RA
It is composed of a computer system composed of M, ROM and the like.

First communication terminal device 1 and second communication terminal device 2
Are connected to each other via a transmission line 5 composed of the Internet or the like so as to be able to transmit and receive each other by packets. From the first communication terminal device 1 to the second communication terminal device is audiovisual data is transmitted is separate jurisdiction to the packet, the time the agent A ₁ in each of its packet, A _2, ..., A _m-1,
Am is added. The time agent is time information exchanged between the encoder 3 of the first communication terminal device 1 and the decoder 4 of the second communication terminal device 2 using a packet.

More specifically, as shown in FIGS.
Time information indicating the time at which the packet was transmitted from the encoder 3 to the decoder 4, time information indicating the time at which the decoder 4 received the packet, time information indicating the time at which the packet was transmitted from the decoder 4 to the encoder 3,
It comprises time information indicating the time at which the encoder 3 received the packet.

As shown in FIG. 2, the encoder 3 is provided with registers TA (n) to TD (n) for storing the time information of the time agent, respectively. Register R for storing the time information of the time agent
A (n) to RD (n) are provided.

FIGS. 3 and 4 show the packet structure. FIG. 3 shows the packet structure when there is audio / video data. FIG. 4 shows the packet structure when there is no audio / video data. Shows the structure. In FIGS. 3 and 4, "carry" is a part for storing information indicating a change in date.

Next, the operation performed by the encoder 3 of the first communication terminal device 1 will be described. The encoder 3 is shown in FIG.
6 and the delay state detection process shown in FIG. 6 are performed in parallel. First, the encoding process will be described. When the encoding process is started, first, as shown in FIG. 5, initialization is performed (S10). In the initialization, predetermined variables are initialized. That is, “R0” is set as an initial value for the compression ratio R, and “0” is set as an initial value for a variable DT (hereinafter, referred to as delay time DT) indicating a delay time. The delay time DT will be described later.

When the initial setting (S10) is completed, first, the delay time DT is read (S20), and it is determined whether the value is smaller than "0" (S30). "Delay time DT
If <0 ”(meaning that transmission is delayed) (S30: YES), in that case, the compression rate R is set based on the delay time DT (S40). R of encoder 3
In the OM, the correspondence between the delay time DT and the compression ratio R is stored as a table, and the compression ratio R is determined by referring to this table. Then, compression encoding is performed at the determined compression ratio R to create audio / video data (S
50).

It should be noted that as the delay time DT falls below 0 (that is, the negative value and the absolute value increases), the delay becomes larger. R is set to be large.
Then, as the compression ratio R increases, the data amount of audiovisual data generated by compression encoding is suppressed.

On the other hand, if the result of determination in S30 is not "delay time DT <0" (NO), then the compression ratio R = "R
It is determined whether it is "0" (S60). Where the compression ratio R =
If it is "R0" (S60), the process proceeds to S50, while if the compression ratio R is not "R0" (S60), the compression ratio is lowered by one step (S70), and thereafter, the process proceeds to S50.

When the compression encoding is completed in S50, the value of the internal clock TR1 in the encoder 3 is obtained (S8).
0), a time agent using the value as time information is created and added to the audiovisual data (S90). In addition,
Here, the time information ~ of the time agent is blank. In S80, the value of the obtained internal clock TR1 is substituted for TA (0) only at the time of the first execution.

Next, a packet transmission interval PT is determined by referring to a predetermined table based on the compression ratio R (S100). It should be noted that the larger the compression ratio R, the smaller the encoded data (that is, audio-video data) generated, and thus the longer the transmission interval PT. On the other hand, when the compression ratio R is small, the amount of encoded data increases, so that the transmission interval PT is set short.

Then, the audio and video data and the time agent are packetized, and at the transmission interval PT set at S100, the transmission of the packet is started (S110), and the transmission of the packet is waited for (S120: NO).
Thereafter, when the transmission of the packet is completed (S120: YE
S), the process returns to S20.

On the other hand, when the delay state detecting process shown in FIG. 6 is started, first, as shown in FIG. 6, "0" is set to a counter variable n as an initial setting (S130). Then, it waits until the time agent sent from the decoder 4 is received (S140: NO).

When the time agent is received (S14)
0: YES), and increments the counter variable n by one (S15)
0), and uses the reception time as time information in the register TD.
(N) (S160). In addition, the received time information of the time agent
(N), TB (n) and TC (n) (S17)
0).

Next, it is determined whether "TA (0) = TA (n)" (S180), and "TA (0) = TA (n)".
If (S180: YES), the process returns to S140. That is, if the received time agent is the first one, the change in the transmission time cannot be detected.
Returning to 140, it waits for the next time the agent is received. If the counter variable n is “1” in S180, which means that the received time agent is the first one, the affirmative determination is made in S180, and the process proceeds to S140.

On the other hand, unless “TA (0) = TA (n)” (S180: NO), the delay time DT is calculated by the following equation (S190). DT = ｛TA (n) -TB
(N) {-{TA (n-1) -TB (n-1)}
Regarding the time (transmission time) required to reach the decoder 4 from the encoder 3, the currently received time agent (n) and the previously received time agent (n-1)
Is calculated as the delay time DT.

Thereafter, the registers TA (n), TB
(N), TC (n) and TD (n) are stored in a register TA.
(N-1), TB (n-1), TC (n-1), TD
Substitute (n-1) (S200). Then, the counter variable n is reduced by one (S210), and thereafter, the process returns to S140.

The one decoder 4 performs the time agent return processing shown in FIG. 7 while reproducing audio and video based on the packet received from the encoder 3. When this process is started, first, the counter variable n
(Different from encoder 3) and reset (S3
00) and waits for a packet to be received (S3
10: NO). When the packet is received (S310: Y
ES), a time agent is extracted from the packet (S320), and the time information contained therein is set in the register RA (n) (S330). The packet reception time is set in the register RB (n) with reference to the internal clock TR2 (S340), and the register R
A scheduled transmission time (that is, a time at which the time agent is transmitted to the encoder 3) is set in C (n) (S3).
50).

Next, it is determined whether or not the counter variable n = "0" (S360). Counter variable n = “0”
Is satisfied (S360: YES), RA (n), R
B (n) and RC (n) are set to the time information, time information, and time information of the time agent, respectively (S37).
0). Then, the counter variable n is increased by one (S38).
0), and further packetizes the time agent (S3)
90), and sends the packet to the first communication terminal device 1 (S400). After transmitting the packet, the process returns to S310.

On the other hand, as a result of the determination in S360, if the counter variable n is not "0" (S360: NO), the packet transmission interval PT is calculated by the following equation: PT = RA (n) -RA (n-1) (S410). And the register RA
(N), RB (n), and RC (n) are set to time information, time information, and time information of the time agent, respectively (S420).

Then, it is determined whether or not the counter variable n>"1" (S430). If the counter variable n is not "1" (S430: NO), the counter variable n is increased by one (S440), and The agent is packetized (S450). If the counter variable n> “1” in S430 (YES), the values of the registers RA (n), RB (n), and RC (n) are changed to the values of the registers RA (n−1) and RB (n−). 1) After setting RC (n-1) (S460), the process proceeds to S450.

The time agent packetized in S450 is transmitted to the encoder 3 so as to have the calculated packet transmission interval PT (S47).
0). After transmitting the packet (S470), S31
Return to 0. In the communication system of the present embodiment as described above, the encoder 3 is the time the agent A ₁ having a transmission time of a packet as time _{information, A 2, ..., A m} -1,
And sending toward the A _m to the second communication terminal device 2, the decoder 4 which received this time the agent A _1, A ₂ having a packet reception time as the time information, ..., A _m-1,
It is transmitted to the A _m to the encoder 3. The encoder 3 detects the delay time DT based on the time information and the time information possessed by the time agent, and sets the compression rate R based on the detected delay time DT. I am trying to become.

As described above, even if the transmission delay is large, it is possible to suppress a situation in which audio and video data necessary for reproduction is lacking by reducing the data amount of the audio and video data. Therefore, during decoding, it is easy to obtain a seamless reproduced signal, and it is possible to suppress skipping of sound and interruption of video.

Further, in the communication system of the present embodiment, the delay state in the transmission path can be grasped even if the packet transmission interval is not constant. The following is a summary of the correspondence between the present embodiment and the claims. That is, the compression rate corresponds to the "coding rate" in the claims, the audiovisual data corresponds to the "coded data" in the claims, and the time agent corresponds to the "time difference information (delay state information)" in the claims. are doing.

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and can take various forms. For example, in the above-described embodiment, as the delay state information, time difference information (more specifically, a pair of time information indicating “generation time and arrival time”) capable of calculating the time difference between the generation time and the arrival time has been described as an example. However, the present invention is not limited to these, and if the transmission interval is constant, “interval of packet arrival”, “number of packet arrivals per unit time”, or the like may be used as delay state information. .

If the transmission interval is constant, the "transmission delay" can be detected by a change in the packet arrival interval or a change in the number of packets arriving per unit time. Such a mode is based on the premise that packets are transmitted at fixed time intervals from the transmission side, but the delay state can always be grasped regardless of the presence or absence of audio / video data to be transmitted, and the transmission path delay state A corresponding compression ratio can always be set. Therefore, from the state where there is no need to transmit audio / video data,
Even when transmission is newly started, skipping of sound and interruption of images can be promptly suppressed. The "interval of packet arrival" and "the number of packets arriving per unit time" can be measured by various means such as a counter including a flip-flop.

In the above embodiment, the first communication terminal device 1 and the second communication terminal device 2 have been described as being connected via the transmission path 5 composed of the Internet or the like, but the present invention is not limited to this. is not. Further, a transmission path when transmitting from the first communication terminal apparatus 1 to the second communication terminal apparatus 2 and a transmission path when transmitting from the second communication terminal apparatus 2 to the first communication terminal apparatus 1 are different networks. It may be, or may be a common network.

In the above embodiment, the description has been made assuming that the audio data and the video data are transmitted only in one direction from the first communication terminal device 1 to the second communication terminal device 2.
However, it is not limited to this. For example, as shown in FIG. 8, a first audio / video signal transmitting / receiving apparatus 101 and a second audio / video signal transmitting / receiving apparatus 102 are provided with encoders 103 and 104 similar to the encoder 3 and the decoder 4 of the above embodiment.
And the decoders 105 and 106 may be provided respectively.

In the configuration shown in FIG. 8, the video / audio signal is transmitted / received by the encoder 103 of the first audio / video signal transmission / reception apparatus 101 as a packet D1 including the audio / video data and the time agent via the Internet. It is sent to the decoder 106 of the device 102. The audio / video data transmitted in the packet D1 is output as a reproduced audio / video signal, and the time agent is returned to the encoder 103 as a packet DT1.

Similarly, the audio / video signal is sent to the decoder 105 of the first audio / video signal transmitting / receiving apparatus 101 together with the time agent as a packet D2.
04 is returned. A reproduced audio / video signal is output from the decoder 105.

In the configuration shown in FIG. 8, the network router determines the packet transmission / reception direction and transmits / receives data to / from the Internet using one port. However, it is not always necessary. Absent. When the audio / video signal transmission / reception device multiplexes transmission / reception packets of the audio / video signal, a network router is not required.

In the configuration shown in FIG. 8, when returning the time agent from the decoder 106 of the second audio / video signal transmitting / receiving apparatus 102 to the encoder 103 of the first audio / video signal transmitting / receiving apparatus 101, a packet carrying audio / video data is used. It is efficient and preferable to use D2.

In the above embodiment, the compression rate is changed in order to change the coding rate.
The present invention is not limited to this, and the sampling frequency, the number of quantization, the spatial resolution, the number of video frames / second, the gradation, and the like may be changed.

[Brief description of the drawings]

FIG. 1 is an explanatory diagram illustrating a schematic configuration of a communication system.

FIG. 2 is an explanatory diagram showing a correspondence relationship between a time agent and registers of an encoder and a decoder.

FIG. 3 is an explanatory diagram showing a packet structure when audio / video data exists.

FIG. 4 is an explanatory diagram showing a packet structure when there is no audio / video data.

FIG. 5 is a flowchart illustrating an encoding process performed by an encoder.

FIG. 6 is a flowchart illustrating a delay state detection process performed by an encoder.

FIG. 7 is a flowchart showing a time agent return process performed by the decoder.

FIG. 8 is an explanatory diagram showing another embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... 1st communication terminal device 2 ... 2nd communication terminal device 3, 103, 104 ... encoder 4, 105, 106 ... decoder 5 ... transmission line 101 ... 1st audio-video signal transmission / reception device 102 ... 2nd audio-video signal transmission / reception device TR1, TR2: Internal clock

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) MM01 MM08 MM24 MM39 NN22

Claims (4)

[Claims] A first communication terminal device for generating encoded data corresponding to an original signal at a predetermined encoding rate, dividing the encoded data into packets, and transmitting the divided packets to a transmission path; And a second communication terminal device for decoding encoded data included in the arrived packet into a reproduction signal when the packet arrives via the communication device, wherein the second communication terminal device The first communication is configured to generate delay state information corresponding to a delay state of packet transmission on the transmission path based on the arrival timing, and to transmit the delay state information to the first communication terminal device. The terminal device sets the encoding rate based on the transmitted delay state information,
A communication system configured to change a data amount of the encoded data to be transmitted. 2. The first communication terminal device is configured to record a generation time in each packet when each packet is generated, and wherein the second communication terminal device records the generation time in each packet. 2. The method according to claim 1, wherein the generation time and the arrival time of the packet are acquired, and time difference information corresponding to a time difference between the generation time and the arrival time is generated as the delay state information.
A communication system as described. 3. The first communication terminal device is configured to generate a packet at a preset time interval and transmit the packet to the transmission path, regardless of the presence or absence of encoded data to be transmitted in the packet, The communication system according to claim 1, wherein the second communication terminal device generates arrival frequency information indicating a packet arrival frequency as the delay state information. 4. The first and second communication terminal devices are communicably connected to each other by a packet, and the second communication terminal device transmits the delay state information to the second communication terminal. The communication system according to any one of claims 1 to 3, wherein the communication system stores the packet in a packet transmitted from the device to the first communication terminal device and transmits the packet.