JP2002319971A - Packet data repeater and distributing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To shorten reproduction delay by reducing the buffering quantity of AV data needed by a reproduction start in a receiving side in the transmission of a plurality of pieces of continuous media data such as AV data about a packet data repeater and a packet data distributing device. SOLUTION: An RTP(real-time transport protocol) packet where a plurality of pieces of continuous media data are stored is extracted from a received packet, an RTP packet filter 1-1 distributes RTP packets having a synchronous relation in each media type and the distributed RTP packets are stored in individual RTP packet buffers 1-2 and 1-3. An RTP packet scheduler 1-4 calculates a transmission priority of each RTP packet having a synchronous relation on the basis of synchronous reproduction information attached to the RTP packet of each of media, an encoding rate and a transmission line band and outputs each RTP packet to a packet output queue 1-5 in the order of the priority. The packet output queue 1-5 transmits each RTP packet to the network of a receiving terminal side according to the priority.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a packet data relay device and a distribution device, and more particularly to a method for transmitting continuous media data such as voice and moving images over an IP (Internet Protocol) network such as the Internet or an intranet. , A packet data relay device and a distribution device in which the reproduction delay is minimized.

[0002]

2. Description of the Related Art With the recent development of Internet / intranet technology, TCP / IP (Transmission Control
The number of applications for transmitting AV (Audio Video) data in real time using the ol Protocol / Internet Protocol is increasing.

As means for transmitting continuous media data such as real-time AV data in a communication environment based on the TCP / IP protocol, a connectionless type protocol such as UDP (User Datagram Protocol) is used.
RTP (Real-time Transport Protocol; IETF RFC1889
Means for transmitting AV data and the like by using a protocol referred to as “specified” is known. The RTP protocol is a transport layer protocol for transmitting and receiving AV data and the like in real time.

[0004] Here, MPEG is used as a moving picture coding method.
(Moving Picture Experts Group) A moving picture coding method that combines intra-frame coding and inter-frame predictive coding typified by a standard method will be described. 2. Description of the Related Art In a moving picture coding method in which intra-frame coding and inter-frame predictive coding are combined, coding is generally performed with a difference in the information amount of each frame (reproduction unit data).

[0005] Normally, an intra-coded frame (hereinafter, referred to as "frame")
It is called "I picture". ) Is assigned a larger amount of information than an inter-frame coded frame (hereinafter, referred to as a “P picture”). In particular, in order to prevent image quality from deteriorating while encoding at a low bit rate, it is necessary to allocate a sufficient amount of information to the I-picture and allocate an amount of information corresponding to the P-picture. Therefore, in some cases, it is necessary to reduce the frame rate.

Here, when the transmission rate of the network is high, there is no problem. However, when the coding rate is almost equal to the transmission rate of the network, if the difference in the amount of information allocated between the pictures becomes extremely large, the reproduction will not be performed. It causes an increase in delay. This will be described below.

If the encoding rate is equal to the transmission rate of the network, and the decoding of the image information is started at the point of time when the reception side has simply received the data of the first frame, the reproduction of the image information is started. , The receiving buffer underflows or overflows.

That is, in the case of video data, once the reproduction is started from a certain frame, the timing for displaying the next frame is determined. However, by the time, the data necessary for reproducing the frame can be received. Need to be guaranteed. It must also ensure that the receive buffer does not overflow by then.

[0009] A video coding format based on MPEG (hereinafter, referred to as MPEG)
Video Elementary Stream format, for short "VideoE
"S format". In), the transmission side encoder adds vbv_delay information to the beginning of each frame in advance and transmits the frame. The decoding timing of each frame assumed on the encoder side is determined on the reception side (decoder side).
Can be grasped from the vbv_delay information.

[0010] The vbv_delay information indicates the amount of storage in the virtual input buffer of received data required before the start of reproduction so that all data necessary for frame reproduction can be received at the time of frame reproduction on the receiving side. It is a numerical value expressed as a reproduction waiting time converted by a bit rate of a transmission path band and a 90 kHz clock unit.

[0011] Alternatively, the decoding timing of each frame is determined based on MPEG-1 system, MPEG-2P
By using a multiplexing format such as S, MPEG-2TS, and the like, the receiving side refers to the system clock reference signal in the stream and the decoding time stamp / presentation time stamp information added to each picture, so that the VideoES Even if the layer is not interpreted, the decoding and reproduction timing assumed on the encoder side can be similarly grasped.

[0012] Here, vbv of MPEG VideoES
The concept of the _delay information will be described with reference to FIG. This figure shows how the data occupancy of the receiving buffer on the decoder side varies with the reproduction of each frame. First, when reception of video data is started, the occupation amount of the reception buffer increases in proportion to the passage of time.

Each frame stores video data in the unit of reproduction. At the beginning of each frame, the buffer occupancy to be received when decoding the frame is represented by vbv_
It is stored as delay information. Actually, this reproducible reception information amount is expressed as reproduction waiting time information in units of 90 KHz clock. Therefore, the buffer occupancy VBVn for decoding the frame n is obtained by the following (Equation 1). VBVn = (bit rate × vbv_delayn) / 90000 (formula 1)

When the reception buffer occupancy reaches VBVn calculated by the above (Equation 1), the data of frame n is extracted from the reception buffer and decoded / reproduced. This makes it possible to continuously reproduce each frame without causing the receiving buffer to underflow or overflow.

However, as can be seen from FIG. 7, when the information amount of one frame is extremely large, the amount of data to be received before the reproduction of this frame starts increases, and as a result, the delay time until the start of reproduction increases. Resulting in. This is why the reproduction delay increases when there is a large difference in the information amount of each frame.

Since the vbv_delay information originally indicates a waiting time from when the receiving terminal receives the vbv_delay information to when a frame associated with the vbv_delay information is reproduced, the data cannot be received at the assumed bit rate. If so, the reproduction may be started after waiting for the time indicated by the vbv_delay information.

On the other hand, such problems are unlikely to occur with audio data. This is because audio data is generally encoded in units of a fixed sample number / fixed length frame. Under such conditions, it is not necessary to deal with a bias in the amount of information as in MPEG video, and as shown in FIG. 8, playback is started with data of two or more frames simply accumulated in the reception buffer. Then, continuous reception and reproduction can be performed without underflow or overflow of the reception buffer.

Next, audio and video are separated by RT.
An environment transmitted in a P session and passing through a network having a plurality of different transmission bands will be described. In the following description, video data is MPEG-1,
It is assumed that the audio data is encoded by an encoding method using vbv_delay information such as 2, 4 or the like, and the audio data is encoded in fixed-length frame units.

For example, as shown in FIG.
And the receiving terminal 9-3 with the relay device 9-2 interposed therebetween.
It is assumed that there are two networks A and B. Here, the transmission bandwidth of each of the networks A and B is defined as Rat
Let eA, RateB (bps). The relay device 9-2 is various devices having a network relay function, such as a router device, a dial-up access server, or a switching hub.

The bit rate of video data is set to RV (bp
s), the bit rate of the audio data is set to RA (bp
s), real-time transmission is possible if RateB ≧ (RV + RA) (Expression 2) is satisfied.

A recent router / switch device is a VoIP (Voice ov) that passes voice (Voice) over an IP network.
er IP) In order to cope with services and the like, there is one equipped with a function of relaying a specific RTP packet with priority. Generally, these relay devices have a function of allocating a part of the network bandwidth for a specific RTP packet. Therefore, in the network B, the reserved band portion is transferred to the network Bn (n = 1,
2, ...).

[0022]

In a case where RTP packets of a plurality of continuous media data having a synchronous relationship such as AV data are relayed to a network Bn, the transmission band of the network Bn is not very wide, When the relay device transmits each RTP packet of each media data to the network Bn without performing the optimal scheduling, the buffering until the receiving terminal side can reproduce is performed if the relay device transmits the RTP packet of each media data to the network Bn. This causes a problem that the reproduction delay time increases.

For example, even if only data that can be reproduced with respect to video data is received, if audio data synchronized with the data is not received at that time,
Video data cannot be played. Conversely, even if audio data can be received in a reproducible amount, the same applies to the case where video data synchronized with the audio data has not been received, and the reproduction delay time increases until the reception of both data is completed.

An Internet service provider (IS
In a relay device such as P), a line with a plurality of users is connected to a backbone network having a wide band owned by itself, and an Internet communication service is provided. When this is applied to the configuration example of FIG. 9, the network A corresponds to a backbone network of an Internet service provider (ISP), and the network B corresponds to a connection line to each user.

Normally, when the relay apparatus transmits AV data, the transmission is performed while managing the timing using the reproduction timing information added to the AV data. For example, when transmitting AV data using the RTP protocol, the RTP header stores the reproduction time information of each frame of the AV data stored in the RTP packet. The transmission timing is determined so that the transmission interval becomes equal to the original reproduction interval.

The transmission of AV data from the relay apparatus is performed when each frame reaches the transmission timing and the bandwidth of the network B is vacant at that time. However, when the transmission interval is managed only by the reproduction time of the frame as described above, if there is a large difference in the information amount of each frame such as MPEG video, the transmission data increases in a burst when transmitting a frame having a large amount of information. And a long transmission delay occurs.

On the other hand, assuming that the audio data has a fixed frame length, the relay device 9-2 transfers video data with higher priority than audio data when relaying the received RTP packet to the network B. Is easy to occur. In such a case, the receiving terminal is likely to underflow the audio data reception buffer.

According to the present invention, in transmission of a plurality of continuous media data such as AV data using a network typified by the Internet / intranet using a TCP / IP protocol, a network in which a packet storing AV data or the like passes is used. Under the condition that the transmission band is narrow enough to slightly exceed the encoding rate of the media data, the buffering amount of AV data required before the reproduction is started on the receiving side is reduced as much as possible to reduce the reproduction delay. The purpose is to:

[0029]

According to the present invention, there is provided a packet data relay apparatus comprising: (1) means for extracting a packet storing a plurality of continuous media data from a received packet; Synchronous playback information extracting means for extracting synchronous playback information added to the continuous media data; encoding rate information identifying means for acquiring coding rate information of the media data having a synchronous relationship; Transmission path band information grasping means for grasping band information of a transmission path to a terminal, transmission scheduling means for calculating transmission priority of a received packet including media data having a synchronous relationship, and transmitting a received packet to a receiving terminal. Transmitting means for transmitting, wherein the transmission scheduling means transmits to the packet of the media data having the synchronous relationship. Then, based on the synchronous reproduction information, coding rate information, and transmission path band information of the continuous media data, a transmission priority of each packet of the media data in the synchronous relationship is calculated, and the transmitting unit determines the priority. According to the order, packets of media data having a synchronous relationship are transmitted to the receiving terminal.

Also, (2) time stamp information added to each reproduction unit data of each media data is used as the synchronous reproduction information, or (3) the amount of reproducible reception information per reproduction unit data of each media data. Or (4) using reproduction waiting time information for each reproduction unit data of each media data.

Further, the packet data distribution apparatus of the present invention provides (5) means for inputting a plurality of continuous media data, means for packetizing each of the input media data, and synchronization between the individual media data. A synchronous relation grasping means for grasping, synchronous reproduction information extracting means for extracting synchronous reproduction information added to the continuous media data, and a coding rate information grasping means for grasping coding rate information of each media data having a synchronous relation. Transmission path band information grasping means for grasping band information of a transmission path to a receiving terminal, transmission scheduling means for calculating transmission priority of media data packets having a synchronous relationship, and transmitting the media data packet to the receiving terminal. Transmitting means for transmitting, wherein the transmission scheduling means transmits packets of the synchronous media data. On the basis of the synchronous reproduction information of the continuous media data, the encoding rate information, and the transmission path band information, the transmission priority of each packet of the media data in the synchronous relationship is calculated, According to the priority, packets of media data having a synchronous relationship are transmitted to the receiving terminal.

[0032]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An RTP packet data relay device of the present invention is a means for ascertaining a transmission band of a network Bn, and outputs audio and video RTP packets to the network Bn according to the transmission speed of the network Bn. It has a function to perform scheduling. As an example, the operation of the present invention will be described with reference to FIG. 1 in the case of relaying one RTP packet stream for each audio / video output from the transmitting terminal.

In FIG. 1, RTP packet filter 1
-1 classifies received RTP packets according to whether they are audio or video, and transfers RTP packets of audio data to audio RTP packet buffer 1-2 and RTP packets of video data to video packet buffer 1-
Accumulate in 3. At this time, the attribute information of each RTP packet is notified to the packet scheduler 1-4.

The information to be notified to the packet scheduler 1-4 is of the following five types. (1) Type of each RTP packet (MPEG-1Layer
r2 audio, MPEG-1 video, etc.) (2) Encoding bit rate of each media data (3) Size of each RTP packet (4) Time stamp information of each RTP packet header (5) RTP packet storing MPEG video data In the case of, vb of the frame stored in the packet
v_delay information; in the case of a packet storing audio data, the size per frame

The RTP packet scheduler 1-4 uses M
Based on the AV stream multiplexing operation defined by the PEG-1 system and the MPEG-2 program stream, each audio R in which the AV stream is stored is stored.
The TP packet and the video RTP packet are multiplexed like the pack data of the MPEG upper layer and output to the RTP packet queue.

More specifically, the packet scheduler 1-4
Compares the value of the virtual system time ST with the queuing time QT of each RTP packet, and gives priority to the RTP packet queue 1 from the RTP packet whose queuing time QT ≦ system time ST (Equation 3). It operates to output to -5.

First, each RTP packet buffer 1-2,
The queuing time QT of the first RTP packet of 1-3 is obtained. At this time, the ratio (RatioTBn) of the transmission rate of the network Bn to the encoding bit rate of the audio / video data is obtained. The bit rate of the video data at the time of encoding is RV, the bit rate of the audio is RA, and the transmission rate of the network Bn is RateBn.
Then, RatioTBn is obtained as follows. RatioTBn = RateBn / (RV + RA) (Equation 4)

Here, RatioTBn is a value of 1 or more. The queuing time QTVi of the video RTP packet including the frame i is obtained as follows. QTVi = (time stamp information of frame i included in RTP header) − (vbv_delay / RatioTBn of frame i) (Equation 5)

Similarly, the queuing time QTAj of the RTP packet of the audio frame j is obtained as follows. QTAj = (time stamp information of frame j included in RTP header) − (frame delay / RatioTBn) (6)

Here, the frame delay means a transmission time per frame at an audio encoding bit rate RA, and corresponds to vbv_delay of MPEG video. This is obtained from the ratio of one frame size of audio data to the audio encoding rate.

The smaller value of the queuing times QTVi and QTAj of each leading RTP packet thus obtained is compared with the value of the system time ST, and the value of the system time ST is equal to or greater than the smaller value. If
The RTP packet at the queuing time is moved from the packet buffer to the packet output queue 1-5.

At this time, the system time ST is advanced by the size of the RTP packet output to the packet output queue 1-5. That is, the output RTP packet size is set to Rat
The system ST is advanced by the time divided by eBn. As a result, it is possible to output the RTP packet so that the receiving terminal always keeps up with the timing to be displayed.

The above RTP packet scheduler 1-4
2 is shown in the flowchart of FIG. First, each RTP
The time stamp data of the first RTP packet data is extracted from the packet buffer, and the queuing time QT is calculated by the above (Equation 5) and (Equation 6) (2-
1). Next, it is checked whether the system time ST has been initialized (2-2). If the system time ST has not been initialized, the system time ST is initialized with the smallest value among the queuing times QT obtained earlier (2). -3). Then, a comparison is made between the system time value ST and the queuing time value QTAj of the audio RTP packet (2-4), and QTAj ≦
In the case of ST, the audio RTP packet Aj is output to the packet output queue, and the RTP packet size /
Increment the system time ST by RateBn (2-
5).

The system time value ST and the video RTP
The size of the packet is compared with the queuing time value QTVj (2-6), and if QTVi ≦ ST, the video R
The TP packet Vi is output to the packet output queue, and the system time ST is incremented by the RTP packet size / RateBn (2-7). If the system time ST is smaller than either of the queuing times QTVi and QTAj, the system time ST is counted up by a predetermined amount until it exceeds one of the queuing times QT (2-8).

The receiving terminal starts decoding the AV data when the video data of the information amount indicated by the vbv_delay information can be buffered in the local receiving memory. By the above-described scheduling transmission in the relay device, the receiving terminal can always maintain the state in which all of the AV frame data necessary for decoding has been completely received at the timing of decoding and reproducing the received AV data. Playback and display can be performed while synchronizing data.

In the above description, the network B
Although the entire band and the reserved band Bn are treated as fixed bands, even when the band fluctuates with time, the transmission speed in the calculation of the queuing time QT in the above (Equation 5) and (Equation 6) is calculated. By reflecting the fluctuating band on the ratio (RatioTBn) as needed, it is possible to determine the optimum priority.

[0047]

FIG. 3 shows an embodiment of a packet data relay apparatus according to the present invention. Here, RTSP (Real T
ime Streaming Protocol (IETF RFC2326) to establish an RTP streaming session,
It is assumed that the relay device is located in the middle of the network between the transmitting and receiving terminals. In the following description, the transmitting terminal is called a server, and the receiving terminal is called a client.

FIG. 3 shows an example of an apparatus for relaying all types of IP packets including RTP packets from the network A to the network B. Therefore, the IP packet input from the network A via the network interface 3-1 is converted to an IP packet other than TCP and UDP.
Packet, TCP IP packet stream, and UD
IP packet filter 3 for sorting to P IP packets
-2.

Further, there is provided a UDP packet filter 3-3 for reconstructing a UDP packet output from the IP packet filter 3-2 and outputting only a packet including an RTP packet to the RTP packet filter 3-4. Further, in order to grasp which RTP packet among the RTP packets flowing from the network A to the network B is transmitting the mutually synchronized media data, the TCP of the TCP flowing from the network A to the network B is determined. R
RTSP monitor section 3-5 for monitoring TSP communication
Is provided.

FIG. 4 shows the RTSP between the server and the client.
The example of the communication content is shown. This shows an example of communication called a DESCRIBE method in the RTSP protocol. When a DESCRIBE request of the RTSP protocol as shown in FIG. 2A is transmitted from the client to the server, the corresponding (b) in FIG. ) Is sent from the server to the client.

In this example, the client
r. example. com for a server called ne
ws. In the response from the server side, information on the format in which the content is transmitted is described in the format shown in the line starting with m =.

In the response example shown in FIG.
The line starting with “audio” indicates that the medium of the payload type = 14 using the RTP protocol is port 210
10 shows an example of transmission to 10 addresses. According to RFC 1890 of the Internet-related technical standardization document, since the payload type = 14 is assigned to MPEG audio, the media type can be determined from this information.

The line starting with m = video indicates that video data of payload type = 98 is transmitted to port 49170 in synchronization with the above audio media. In RFC 1890 of the Internet-related technical standardization document, the payload type = 98 is defined as a payload type that can be dynamically assigned, and the actual media type is the following a.
From the character string “MPV” written on the line starting with “=”, M
PEG-1 or MPEG-2 Video is identified.

The client to which the response from the server is sent is identified from the destination IP address of the IP packet containing the response message. Based on the destination IP address / port number and the payload type information, the RTP packet filter 3
-4 classifies the synchronous media included in the session X addressed to the destination client M, and buffers the media in the RTP packet buffers 3-6x1 to xj for each media.

In FIG. 3, each RTP packet buffer 3-6 has a plurality of (m) RTP packet queues corresponding to each of the RTP packet schedulers 3-71 to 3-7m and a media type (j). Provided. Any one of the RTP packet schedulers 3-71 to 3-7m is assigned to the session X addressed to the client M of the transmission destination.
One is selected and assigned, and the RTP packet filter 3
-4, each RTP packet buffer 3-6x1 to xj of the selected one RTP packet scheduler 3-7x
Each RTP packet is distributed to each media.

Since the destination port number of the RTP packet may be changed by communication using a method performed between the client and the server subsequent to the DESCRIBE method, the relay apparatus also monitors this. There is a need. RTP packet buffer 3-6
The RTP packets stored in x1 to xj are taken out in priority order according to the system time ST managed in the RTP packet scheduler 3-7x, and input to the corresponding RTP packet queue.

Here, the bandwidth of the network B is allocated for each RTP session, and the RT for the client M is
It is assumed that the band Bm is allocated to the P packet. R
Each RTP packet addressed to the client M input to the TP packet queue is output as needed when the bandwidth Bm of the network B is available. As a result, it is possible to grasp the synchronous relationship between a plurality of RTP sessions established by the RTSP protocol and perform packet relay scheduling according to the available bandwidth Bm.

Further, an RTSP proxy server is separately provided, and the analysis of the RTSP communication is performed by the RTSP proxy server.
It can be operated by receiving from a P proxy server. In this case, as shown in FIG.
Instead of not providing the TSP monitor function, the means 5-1 for receiving synchronous RTP packet information required for filtering in the RTP packet filter 3-4 from a device having an RTSP communication monitor function such as another RTSP proxy server is used. Required.

FIG. 6 shows an embodiment of the packet data distribution device of the present invention. In this embodiment, unlike the above-described relay device, instead of receiving and relaying a packet storing AV media data from the network A, the RTP transmission request unit 6-1 receives an external request, A synchronized from the AV media storage unit 6-2 in response to the request
V media data is read, and the RTP packet generator 6
-3, the RTP packets are distributed and stored in separate RTP packet buffers 6-41 to 6-4j according to the media type, and transmitted according to the bandwidth of the network B by the RTP packet scheduler 6-5. The RTP packets are delivered to the network B in priority order according to the schedule.

In the RTP transmission request receiving section 6-1,
Upon receiving the following information, the distribution apparatus converts the media data corresponding to the received request into an RTP packet and transmits the RTP packet. (1) Designation information of media data to be transmitted (assuming that the request transmitting side already knows what is stored in the AV media storage unit 6-2.) (2) Distribute each media data Destination IP address /
port number

Also, information about a network band available for transmitting the requested media data is
It is separately input from a network operation management unit not shown. On the basis of these information, a plurality of A
When transmitting the V media data, the RTP packet scheduler 6-5 performs the same scheduling as in the above-described embodiment, and stores the RTP packets in the RTP packet queue 6.
-6, and distributes it to the network B to the request sender.

In the embodiment shown in FIG. 6, the RTP packet is generated by the RTP packet generator 6-3RT in the distribution device.
Although the P packet generation unit 6-3 performs this processing, the AV data is stored in advance in the form of RTP packet data in the AV media storage unit 6-3.
-2, and the distribution device may treat the packet read from the AV media storage unit 6-2 as an RTP packet as it is. In addition, various modifications can be made without departing from the gist of the present invention, such as adding designation information for designating a transmission start position as the content of a transmission request input from the outside.

(Supplementary Note 1) A means for extracting a packet storing a plurality of continuous media data from a received packet, a means for ascertaining a synchronous relationship between individual pieces of media data, and a means added to the continuous media data Synchronous reproduction information extracting means for extracting synchronous reproduction information, coding rate information grasping means for grasping coding rate information of media data in a synchronous relationship, and transmission for grasping band information of a transmission path between the receiving terminal Path bandwidth information grasping means, transmission scheduling means for calculating the transmission priority of a received packet including media data having a synchronous relationship, and transmission means for transmitting the received packet to a receiving terminal, the transmission scheduling means comprising: , For the synchronous media data packet, synchronous playback information of the continuous media data, Based on the coding rate information and the transmission path band information, the transmission priority of each packet of the media data in the synchronous relationship is calculated, and the transmitting unit determines the packet of the media data in the synchronous relationship in accordance with the priority. A packet data relay device for transmitting to a receiving terminal. (Supplementary Note 2) The packet data relay device according to Supplementary Note 1, wherein time stamp information added to each reproduction unit data of each media data is used as the synchronous reproduction information. (Supplementary Note 3) The packet data relay device according to Supplementary Note 1, wherein an amount of reproducible reception information for each reproduction unit data of each media data is used as the synchronous reproduction information. (Supplementary Note 4) The packet data relay device according to Supplementary Note 1, wherein the synchronous reproduction information uses reproduction waiting time information for each reproduction unit data of each media data. (Appendix 5) RTSP (Real
(Supplementary note 1) characterized by comprising means for analyzing packet information of Time Streaming Protocol (communication) communication, and grasping the destination IP address and port number of the packet storing the synchronous media data based on the RTSP packet information. Packet data relay device. (Supplementary Note 6) A means for inputting a plurality of continuous media data, a means for packetizing each input media data, a synchronization relation grasping means for grasping a synchronization relation between the individual media data, and an addition to the continuous media data Synchronous reproduction information extracting means for extracting the synchronized reproduction information, encoding rate information grasping means for grasping the encoding rate information of each media data having a synchronous relationship, and bandwidth information of a transmission path between the receiving terminal and the receiving terminal. Transmission path bandwidth information grasping means for grasping, transmission scheduling means for calculating transmission priority of media data packets having a synchronous relationship, and transmission means for transmitting the media data packet to a receiving terminal; Is synchronous reproduction information of the continuous media data with respect to the packet of the media data having the synchronous relationship. , The transmission priority of each packet of the media data having the synchronous relationship based on the coding rate information and the transmission path band information, A packet data distribution apparatus for transmitting a packet data to a receiving terminal. (Supplementary Note 7) Means for inputting a plurality of packetized continuous media data, means for grasping a synchronization relationship between the inputted individual media data, and synchronous reproduction information added to the continuous media data Synchronous playback information extracting means, encoding rate information grasping means for grasping coding rate information of each media data having a synchronous relationship, and a transmission path band for grasping band information of a transmission path between the receiving terminal. Information grasping means, transmission scheduling means for calculating the transmission priority of media data packets having a synchronous relationship, and transmission means for transmitting the media data packets to a receiving terminal, wherein the transmission scheduling means comprises: , The synchronous reproduction information of the continuous media data, the encoding rate information, Transmission priority of each packet of the media data having the synchronization relationship is calculated based on the transmission path bandwidth information, and the transmitting means transmits the packet of the media data having the synchronization relationship to the receiving terminal according to the priority order. A packet data distribution device characterized by the above-mentioned.

[0064]

As described above, according to the present invention,
Internet using TCP / IP protocol /
In an IP packet-based AV data transmission represented by an intranet communication, a relay device or a distribution device is provided under a condition that there is no large difference between a coding rate of a synchronized AV medium and a network transmission band of a packet storing the AV data. The device side schedules the transmission order of each AV packet according to the information amount of each frame and the rate of the transmission band, so that the receiving side minimizes the amount of buffering of AV data required before the start of reproduction. Thus, it is possible to further reduce the reproduction delay.

In particular, when it is desired to give priority to the resolution over the frame rate with respect to the video data to be transmitted, a large variation tends to occur in the data amount of each frame.
In such a case, audio data synchronized with the video data can be transmitted with lower delay.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of a packet relay device of the present invention.

FIG. 2 is an operation flowchart of the packet scheduler of the present invention.

FIG. 3 is a configuration diagram of an embodiment of a packet relay device of the present invention.

FIG. 4 is a diagram showing an example of RTSP communication contents.

FIG. 5 is a configuration diagram of an embodiment of a packet relay device of the present invention provided with an RTSP communication monitor function externally.

FIG. 6 is a configuration diagram of an embodiment of a packet data distribution device of the present invention.

FIG. 7: MPEG video data reception buffer and vb
It is explanatory drawing of v_delay information.

FIG. 8 is an explanatory diagram of audio data reception buffers and reproduction conditions.

FIG. 9 is a configuration diagram of an RTP packet transmission / reception system including a packet relay device.

[Explanation of symbols]

 1-1 RTP packet filter 1-2 Audio RTP packet buffer 1-3 Video packet buffer 1-4 Packet scheduler 1-5 Packet output queue

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5K030 GA02 GA06 HA08 HB02 HB15 HB16 HB21 JA11 JL07 KA03 LA03 LA15 LE05

Claims (5)

[Claims] 1. A means for extracting a packet storing a plurality of continuous media data from a received packet; a means for ascertaining a synchronous relationship between individual media data; and a synchronous reproduction added to the continuous media data. Synchronous reproduction information extracting means for extracting information; coding rate information grasping means for grasping coding rate information of media data in a synchronous relationship; and a transmission path band for grasping band information of a transmission path between the receiving terminal. Information grasping means, transmission scheduling means for calculating a transmission priority of a received packet including media data having a synchronous relationship, and transmission means for transmitting a received packet to a receiving terminal, wherein the transmission scheduling means comprises: Synchronous playback information and coding level of the continuous media data are The transmission unit calculates the transmission priority of each packet of the media data having the synchronous relationship based on the transmission information and the transmission path band information, and the transmitting unit transmits the packet of the media data having the synchronous relationship according to the priority. A packet data relay device, which transmits the packet data to a device. 2. The packet data relay device according to claim 1, wherein time stamp information added for each reproduction unit data of each media data is used as the synchronous reproduction information. 3. The packet data relay device according to claim 1, wherein an amount of reproducible reception information for each reproduction unit data of each media data is used as the synchronous reproduction information. 4. The packet data relay device according to claim 1, wherein the synchronous reproduction information uses reproduction wait time information for each reproduction unit data of each media data. 5. A means for inputting a plurality of pieces of continuous media data, means for packetizing each piece of input media data, means for ascertaining a synchronization relationship between individual pieces of media data, Synchronous reproduction information extracting means for extracting the added synchronous reproduction information; coding rate information grasping means for grasping coding rate information of each media data having a synchronous relationship; band information of a transmission path between the receiving terminal Transmission channel bandwidth information grasping means for grasping the transmission schedule, transmission scheduling means for calculating the transmission priority of media data packets having a synchronous relationship, and transmission means for transmitting the media data packet to a receiving terminal. Means for synchronously playing back the continuous media data with respect to the packet of the media data in the synchronous relationship; Information, coding rate information, and transmission path band information, and calculates the transmission priority of each packet of the synchronous media data. A packet data distribution device for transmitting a packet to a receiving terminal.