JPH11164270A - Method and device for transmitting video data using multi channel - Google Patents

Abstract

PROBLEM TO BE SOLVED: To start the decoding of video data without large delay even if buffer memory of a receiving end has a small capacity by storing the video data that is undergone hierarchical encoding in a sending end memory area in a frame unit, sending it to plural channels at each transmission rate for every hierarchy that is defined by the generation information volume and at transmission time allocation timing and temporarily storing and sending it in a receiving end memory area. SOLUTION: Generation information volume of each frame is held by a file 1 and encoded video data are stored in disks 4 to 6 on different hierarchies. A buffer parameter deciding device 2 calculates the transmission bit rate of each channel, the initial storage quantity and maximum quantity of a receiving end and the difference of sending start time between channels based on these information and notifies them to reading and sending devices 10 to 12, a receiver and a sending timing controller 3 respectively. Next, the device 3 controls switches to be sequentially turned on and video data are sent from the disks 4 to 6 to the receiving end.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to video broadcasting using a network, and more particularly, to the Internet and LA.
The present invention relates to a video data transmission method and an apparatus for transmitting, decoding, and displaying video data transmitted from a transmitter in real time using a video data transmission system such as an N or ATM network.

[0002]

2. Description of the Related Art At present, point-to-point communication in which video data is transmitted through a single channel is mainly used, but video data to be received is transmitted in accordance with the connection line capacity and decoding capability of a receiver. For the purpose of controlling the transmission speed and quality, a method has been proposed in which video data hierarchically encoded using a plurality of channels is assigned to each channel for each layer. [(Document 1: Yokota Kubota, Ito, Asami "Experiments on Q ₀ S using hierarchical transfer data in a multi-point communication", 1996 Toshishingakusodai B-834 (199
6)) and (Ref. 2: Sazawa, Takishima, Wada: "A Study of Video Multicasting Method in PSVP Network", IEICE D-11-89 (1997))

FIG. 12 shows a conceptual diagram. In FIG.
Video data is divided into five layers and transmitted by five multicast sessions. At this time, since the terminal station 3 having a small available bandwidth can receive only one layer, a request to participate in the session corresponding to the terminal station 3 is sent to the router 2, and as a result, the router 1 is routed to the router 2. In addition, while the terminal station 1 receives all layers because the available bandwidth is large, the terminal station 2 has a low decoding capability and cannot receive the data in two layers, so that routing is performed by merging those requests. . in this way,
In the model shown in FIG. 12, the network bandwidth is optimally used without waste according to the selection from the receiver.

[0004]

When transmitting video data using a plurality of channels, attention must be paid to the time relationship between the channels. Otherwise, for example, even when it is time to decode, there is a possibility that data necessary for decoding does not arrive. For this reason, the conventional technique uses a method in which a very large buffer is provided on the receiving side and a large amount of data is stored in advance from the start of reception to the start of decoding. Therefore, in addition to the need for a large buffer on the receiving side, there is a disadvantage that delay is increased due to a delay in starting decoding.

[0005] An object of the present invention is to provide a relatively small-capacity buffer memory on the receiving side and start decoding without a large delay, so that transmission and reception of video data on each channel does not occur. It is an object of the present invention to provide a method and apparatus for transmitting video data using multi-channels capable of performing efficient video data transmission in real time.

[0006]

In order to achieve the above object, a video data transmission method using multi-channel according to the present invention comprises the steps of: For transmission over multiple channels individually assigned to the hierarchy,
On the transmitting side, the video data hierarchically encoded into the plurality of layers is stored in a plurality of transmitting memory areas provided for each of the plurality of layers in frame units, and is stored in the plurality of transmitting memory areas. The video data layer-encoded into the plurality of layers stored in frame units is determined according to the individual transmission rate for each layer determined using the amount of information generated per frame and the transmission time allocation timing for each layer. , Transmitted to the plurality of channels, and on the receiving side,
The video data hierarchically coded into the plurality of layers transmitted on the plurality of channels includes a plurality of reception-side memory areas having individual storage capacities for the respective layers determined using the amount of information generated per frame. In accordance with the individual transmission rate for each layer and the transmission time allocation timing for each layer, which is temporarily stored for each layer and is determined using the amount of generated information per frame, the data is read from the plurality of reception-side memory areas. It is provided with a configuration to be synthesized.

Further, in order to carry out the method of the present invention, the transmitting apparatus transmits video data hierarchically encoded into a plurality of layers in consideration of time resolution through a plurality of channels individually allocated to the plurality of layers. A plurality of transmission-side memory areas provided for storing video data hierarchically encoded in the plurality of layers in units of frames for each of the plurality of layers; Buffer parameter determining means for calculating the amount of information generated per frame of the video data hierarchically encoded in the plurality of layers stored in the transmitting side memory area in frame units, and calculated by the buffer parameter determining means. A transmission tag for calculating an individual transmission rate for each layer and a transmission time allocation timing for each layer using the amount of information generated per frame. According to the transmission time allocation timing for each of the layers calculated by the transmission timing control means, the video data hierarchically coded into the plurality of layers from the plurality of transmission-side memory areas in the plurality of layers. A plurality of reading and transmitting means for reading out for each layer and transmitting the data at a plurality of channels individually allocated to the plurality of layers at the individual transmission rate for each layer calculated by the transmission timing control means. It has a configuration.

Further, in order to carry out the method of the present invention, the receiving apparatus transmits video data hierarchically encoded into a plurality of layers in consideration of time resolution through a plurality of channels individually allocated to the plurality of layers. In order to temporarily store the video data hierarchically encoded in the plurality of layers transmitted on the plurality of channels for each of the layers, the receiving side determines the reception side using the amount of information generated per frame. A plurality of receiving memory areas provided so as to have individual required storage capacities for the respective layers, and temporarily storing the video data of a layer to be transmitted earliest among a plurality of layers considering the time resolution. The initial storage amount of one of the plurality of receiving memory areas arranged for storing is determined using the amount of information generated per frame. When the number exceeds the threshold, the read combining unit reads and combines from the plurality of receiving-side memory areas in accordance with the individual transmission rate for each layer determined using the amount of information generated per frame and the transmission time allocation timing for each layer. Means.

[0009]

DETAILED DESCRIPTION OF THE INVENTION The present invention provides a multi-channel C
H _1, CH _2, when applied to a transmission path using a CH _3, 2 frames / S with CH _1, in CH ₁ + CH ₂ 10 frames _{/ S, CH 1 + CH 2} + CH 3 at 30 frames /
S is transmitted, and can be used in a transmission system using multi-session cast as described with reference to FIG.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS As a hierarchical coding method, an international standard method MPE is used.
The difference between the prediction modes in the I, P, and B frames as seen in G is used. Here, an I-frame is an image frame encoded independently. The P frame is an image frame obtained by coding a prediction error when a prediction is made from an I frame or a P frame that is temporally earlier than the encoding target frame. The B frame is an image frame obtained by bidirectionally predicting a temporally past and future I frame or P frame of the encoding target frame and encoding the prediction error. FIG. 1 shows an example of the assignment of the I, P, and B modes of the image frames that are temporally continuous from left to right, and the predicted / predicted relationship between them, as indicated by arrows. From this relationship, the I frame can be decoded by itself, but the decoding of the P frame requires the decoding of the I frame in advance, and the decoding of the B frame requires the decoding of the I and P frames in advance. I understand. Therefore, the I frame is a base layer, P is its higher layer, and B is its higher layer. At this time, I frame alone, I frame
The number of frames to be decoded, that is, the time resolution of the reproduced image can be increased by increasing the number of layers to be decoded in the order of the combination of P and P frames and the combination of I, P, and B frames.

Since the I, P, and B frames are divided into layers as described above, each frame is assigned to a transmission channel as shown in Table 1 below.

[Table 1]

Here, as for the combination of I, P, and B frames, 15 frames are one cycle, and the frame type displayed as the following frame sequence changes over time. IBBPBBPBBPBBPBBIBBPBBP... In this case, it is assumed that the amount of generated information for each frame is known.

The configuration and operation of the transmitter will be described with reference to FIG. First, the generated information amount for each frame is stored in the file 1. Also, video data encoded by the MPEG system is classified by hierarchy, that is, I, P, B
It is stored on the disks 4, 5, and 6 for each frame. Next, based on this information, the buffer parameter determination device 2
In step (1), the amount of information generated for each frame of file 1 is used to calculate the transmission bit rate for each channel and the initial storage amount and the maximum required amount of the buffer on the receiving side by the algorithm described below. In addition, the difference of the transmission start time between the channels is calculated based on the initial accumulation amount. The transmission bit rate from the buffer parameter determination device 2 is read out and transmitted by the read / send devices 10, 11, and 12.
, The initial storage amount and the maximum required amount are notified to the receiver, and the transmission start time difference between the channels is notified to the transmission timing control device 3. Next, the transmission timing control device 3 controls so that the switches 7, 8, and 9 for reading video data from the disks 4, 5, and 6 are sequentially turned on for each channel. Each of the switches 7, 8, 9 is turned on when the transmission start time of the corresponding disk 4, 5, 6 is reached, and the video is transmitted from the disk 4, 5 or 6 connected to the switch 7, 8 or 9 which is turned on. The data is transmitted to the receiving side via the corresponding read / transmit device 10, 11 or 12. From the disk 4, 5 or 6 which has once started reading, the operation is continued until the transmission of all the frames stored on that disk 4, 5 or 6 is completed.
Stay in N state. The reading / sending device 10,
Reference numerals 11 and 12 denote channels CH ₁ and CH ₁ , respectively, according to the transmission bit rate notified from the buffer parameter determination device 2.
Respectively CH _2, CH ₃ unique constant fixed rate predetermined for each reading video data from the disc 4, 5, 6, and sends the channel _{CH 1, CH 2, CH 3} .

The structure and operation of the receiving device will be outlined with reference to FIG. First, the reception buffers 20, 21, and 22 store video data sent from the transmission side, and
To the decryption device 25 via. The control device 23 controls the switch 24 according to the buffer parameters and the decoding timing chart notified from the transmitting side. Decoding device 2
5 operates as a normal video coder because a normal MPEG stream is input as a result of the switch control up to that time. The display device 26 once accumulates the image frames output from the decoding device 25, reads them out in the order in which they should be displayed, and displays them. The detailed operation of the receiving side will be described later. Hereinafter, each device will be described.

[Buffer Parameter Determination Apparatus] Step 1) Determination of Channel Bit Rate The amount of information generated for each frame is totaled for each frame type, and a value obtained by dividing by the display time of all image frames is set as a transmission bit rate for each channel. Channels CH ₁ and CH
Let R ₁ , R ₂ , and R ₃ be ₂ and CH ₃ respectively. Step 2) Determination of buffer parameters As buffer parameters, the initial buffer accumulation amount D ₀ ,
That is, the amount of data stored in the reception buffer before the decoding operation starts on the receiving side and the maximum buffer required amount D
There is a max, that is, the receiving buffer size required on the receiving side. These, after initially determined for D ₀ for each channel, calculated for Dmax.

Since the channel CH ₁ D ₀ has the maximum value satisfying the inequality (1), FIG.
Is calculated according to the flowchart shown in FIG. Here, j is an index that increases every I frame, q _j
Is the amount of information generated in the i-th frame, and R ₁ is the channel CH ₁
, T is one frame interval (0.5 seconds), and N is the number of I frames in all frames.

(Equation 1) Then, Dmax is calculated as the maximum Dmax satisfying inequality (2) based on D ₀ obtained as described above.

(Equation 2) Therefore, it is calculated using the flowchart shown in FIG.

[0017] The channel CH ₂ channel CH ₂ determines the maximum D ₀ that satisfies the following inequality (3). Here, m is an index that increases for each P frame, p _i is the information generation amount of the i-th P frame, R ₂ is the transmission speed of channel CH ₂ , T is the P frame interval (1/10 second), and N is P in all frames
The number of frames, k, is any non-negative integer.

[0018]

(Equation 3) Therefore, the maximum D ₀ according to the flowchart shown in FIG.
Ask for.

[Outside 1]

Next, using D ₀ obtained in this manner,
The maximum Dmax satisfying the following inequality is calculated according to the flowchart shown in FIG.

(Equation 4)

[0020] For channel CH ₃ channel CH _3, obtaining the maximum D ₀ which satisfies the following inequality (5). Here, n is an index that increases for each B frame, n is the information generation amount of the i-th B frame, R ₃ is the transmission speed of channel CH ₃ , and T is B
The frame interval (1/30 second), N is the number of B frames in all frames, and k is any non-negative integer.

(Equation 5) Therefore, the calculation is performed according to the flowchart shown in FIG.

Next, using D ₀ obtained in this manner,
The maximum Dmax satisfying the following inequality is calculated according to the flowchart shown in FIG.

(Equation 6)

[0022] Step 3) The time relationship calculation channels CH ₁ to delays that t ₁ hour after between channels starts transmission channel CH _2, starts to transmit the channel CH ₃ to t ₂ hours after delayed channel CH ₁ . At this time,

(Equation 7) Calculate the times t ₁ and t ₂ . Where D ₀
⁽¹⁾ the value of D ₀ for the channel ^{_{CH 1, D 0 (2)}} D 0 for the channel CH ₂ may, D ₀ ⁽³⁾ is the value of D ₀ for the channel CH _3. Τ is 1/30 second.

As described above, the buffer parameters D ₀ and Dmax for each channel are obtained, and these are notified to the receiving side. Further, parameters t ₁ and t ₂ that determine the time relationship between channels are sent to the transmission timing control device. Further, transmission rates R ₁ ,
R ₂ and R ₃ are notified to the reading / sending device.

The transmission start switch of each channel is controlled based on t ₁ and t ₂ notified from the buffer parameter determination device. That is, the switch 8 is turned on t ₁ seconds after the switch 7 is turned on, and the switch 9 is turned on t ₂ seconds after the switch 7 is turned on.
become. Each switch remains on until the transmission of all image frames is completed.

[Reading / Sending Device] The present device reads image data from a disk in accordance with the transmission rate notified from the buffer parameter determining device and sends it out to a transmission channel. At this time, both processes of the reading / sending device are performed at a fixed speed. Note that while the switch between the disk and the reading / sending device is open, neither reading of the video data nor sending to the channel is performed. Conversely, when the switch is turned on, reading and sending are started from that moment.

[Time Relationship of Decoding Process] As described above, I,
When the P and B prediction modes are used, there is a temporal dependency between the respective frames. For example, as described above, I frames must be decoded earlier than P frames. Due to such temporal dependency, a relationship as shown in Table 2 is established between the display and the decoding process. That is, while the B frame is displayed at the same time as decoding, the I and P frames are each delayed by three time indexes from the decoding to the display. Also, the time from the decoding of the first I frame I ₀ to the decoding of the first P frame P ₀ is 3 time indexes, and the time from the decoding of I _{0 to} the decoding of the first B frame B ₀ is 4 It can be seen that there is a difference in the time index. Here, the time index increases by 1 every 1/30 second.

FIG. 10 shows the decoding start time and the reception start time for each channel. In FIG. 10, D ₀ ⁽¹⁾ / R ₁ , D ₀ ⁽²⁾ / R ₂ , D ₀ ⁽³⁾ / R ₃
Is the time for accumulating the received data in the buffer from the start of the reception to the start of the decoding, since the initial storage amount is the state at the transmission speed.
t ₁ and t ₂ are time differences between transmission start times between channels, that is, time differences between reception start times. K is 1/3
This is a time index that increases by 1 every 0 seconds. t
₁ , t ₂ , D ₀ ⁽¹⁾ / R ₁ , D ₀ ⁽²⁾ / R ₂ , D ₀ ⁽³⁾
From / R ₃ , it can be seen that the decoding start time difference between channels 1 and 2 and the decoding start time difference between channels 1 and 3 follow Table 2.

[0028]

[Table 2]

[Receive Buffer] The basic function of the receive buffer is to store received data, read out the oldest data in order from the moment the switch is closed, and send it to the decoding device. At this time, data for one image frame is sent to the decoding device. Therefore, as shown in FIG. 11, the temporal change of the buffer accumulation amount increases at a fixed rate by the data transmitted at a fixed speed, while the data of one image frame is transferred to the decoding device, so that the instantaneous change occurs. Phenomenon. Here, the reception buffer is empty until the start of reception, and has the size of the maximum required buffer amount notified from the transmission device. Further, it always notifies the control device 23 of the amount of accumulated data.

[Control unit] The transmission device notifies the initial accumulation amount and the maximum required amount of the buffer parameter. Based on this, buffers of the maximum required amount or more are prepared as the respective receiving buffers. Next, when the reception is started, the amount of data stored in the reception buffer is monitored, and the switch 24 is connected to the reception buffer 20 at the moment when the storage amount of the reception buffer 20 becomes D ₀ ^(1). Then, the decoding of the I frame is started. Thereafter, by switching the switches among the reception buffers 20, 21, 22 according to the timing chart shown in Table 2, the decoding process proceeds.

[Decoding device] By the operation of the switch 24,
From the viewpoint of the decoding apparatus, the MPEG video data is input in a regular order, so that a normal MPEG decoding operation may be performed. Since the order of decoding and the order of display are different as shown in Table 2 as in the case of ordinary MPEG, they are adjusted at the time of output.

From the amount of generated information for each frame obtained by actually encoding an image, the transmission rates R ₁ ,
R ₂ , R ₃ , buffer initial accumulation amount D ₀ ⁽¹⁾ ,
D ₀ ⁽²⁾ , D ₀ ⁽³⁾ , required storage amount Dmax ⁽¹⁾ , Dmax
^{(2), Dmax (3)} , and the time difference t ₁ between the channels,
The result of obtaining t ₂ shown in Table 3.

[0033]

[Table 3]

[0034]

According to the present invention, when transmitting video data using a plurality of channels, it is possible to reduce the size of the buffer on the receiving side and to minimize the amount of data that must be initially stored. So that the delay is reduced.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating I, P, and B frames used in the present invention.

FIG. 2 is a block diagram showing an embodiment of a transmitter according to the present invention.

FIG. 3 is a block diagram showing an embodiment of a receiver according to the present invention.

FIG. 4 is a time chart for explaining calculations used in the present invention.

FIG. 5 is a time chart for explaining calculations used in the present invention.

FIG. 6 is a time chart for explaining a calculation used in the present invention.

FIG. 7 is a time chart for explaining calculations used in the present invention.

FIG. 8 is a time chart for explaining a calculation used in the present invention.

FIG. 9 is a time chart for explaining calculations used in the present invention.

FIG. 10 is a time chart for explaining the operation on the receiving side in the present invention.

FIG. 11 is a time chart showing a change in an accumulation amount of a reception buffer on the reception side according to the present invention.

FIG. 12 is a diagram for explaining a conventional video transmission model using multi-session multicast.

[Explanation of symbols]

1 file 2 buffer parameter determining unit 3 sends timing control system 4, 5, 6 disks 7, 8, 9 switches 10, 11, 12 read delivery device CH _1, CH _2, CH ₃ channels 20, 21, 22 receive buffer 23 control Device 24 switch 25 decoding device

Claims (3)

[Claims] In order to transmit video data hierarchically coded to a plurality of layers in consideration of a time resolution through a plurality of channels individually allocated to the plurality of layers, a transmitting side includes: The hierarchically encoded video data is stored in a plurality of transmission-side memory areas provided for each of the plurality of layers in units of frames, and the plurality of layers stored in the plurality of transmission-side memory areas in units of frames. The video data hierarchically encoded in the hierarchy is transmitted to the plurality of channels according to the individual transmission rate for each hierarchy determined using the amount of information generated per frame and the transmission time allocation timing for each hierarchy, On the receiving side, the video data hierarchically encoded into the plurality of layers transmitted on the plurality of channels uses the amount of information generated per frame. The individual transmission rates for each layer, which are temporarily stored for each layer in a plurality of receiving-side memory areas having individual storage capacities for each layer determined according to the A video data transmission method using a multi-channel that is read from the plurality of receiving-side memory areas and combined according to a transmission time allocation timing for each layer. 2. A method for transmitting video data hierarchically coded to a plurality of layers in consideration of time resolution through a plurality of channels individually allocated to the plurality of layers, comprising: A plurality of transmission-side memory areas provided for accumulating layer-encoded video data in frame units for each of the plurality of layers; and the plurality of transmission-side memory areas accumulated in frame units in the plurality of transmission-side memory areas. Buffer parameter determining means for calculating the amount of information per frame of video data hierarchically encoded in a plurality of layers; and using the amount of information per frame calculated by the buffer parameter determining means, Transmission timing control means for calculating an individual transmission speed for each of the layers and a transmission time allocation timing for each of the layers, and the transmission timing control Reading video data hierarchically encoded into the plurality of layers from a plurality of transmission-side memory areas for each of the plurality of layers according to the transmission time allocation timing for each of the layers calculated by the stage; And a plurality of reading and transmitting means for transmitting the data at a plurality of channels individually allocated to the plurality of layers at the individual transmission rates for the layers calculated by the control means. 3. In order to transmit video data hierarchically coded to a plurality of layers in consideration of a time resolution through a plurality of channels individually allocated to the plurality of layers, a receiving side uses the plurality of channels to transmit the data. In order to temporarily store the transmitted video data hierarchically encoded in the plurality of layers for each of the layers, it is necessary to have an individual required storage capacity for each of the layers determined using the amount of information generated per frame. A plurality of receiving-side memory areas provided in the plurality of receiving-side memories arranged to temporarily store the video data of a layer to be transmitted earliest among a plurality of layers in consideration of the time resolution. When an initial storage amount of one of the regions exceeds an initial storage amount determined using the generated information amount per frame, the generated information amount per frame; And a read-out synthesizing means for reading out from the plurality of receiving-side memory areas and synthesizing the read-out synthesizing means according to an individual transmission rate for each layer determined by using the above and a transmission time allocation timing for each layer.