JP2000092049A - Multi-cast system, receiver and multi-cast control method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique for data delay and omission generated by the factor of excessive loads or the like in a network and a client terminal. SOLUTION: A transmitter converts data to be transmitted to packets and adds the continuity information of the packets to the respective packets and this receiver judges the presence/absence of continuity based on the continuity information of the received packets. In the case of judging that the continuity is present, they are stored in a data buffer 36. In the case of judging that the continuity is absent, the data incapable of constituting a decodable minimum unit in the data of the packets received before and after a lost packet are eliminated. In the case that the data of the lost packet can be restored, they are restored and stored in the data buffer 36. The data of the data buffer 36 are decoded 31.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a multicast system in which a transmitting device and a receiving device communicate via a network.

[0002]

2. Description of the Related Art A conventional multicast (broadcast) system is generally introduced in a system in which a transmission path of a network is guaranteed, and is constructed on the assumption that no data is lost. ing. However, there is actually a lack of data, and as a technology to prevent this,
For example, there is a technique described in Japanese Patent Application Laid-Open No. 9-261255.

In the technique described in Japanese Patent Application Laid-Open No. 9-261255, when generating an acknowledgment packet, the acknowledgment number for the data packet received from each member of the multicast group is multicast as one acknowledgment packet. That is, if data is lost, retransmission is performed to prevent packet loss.

Conventionally, there has been little demand for adapting multicast to real-time broadcasting.

[0005]

However, the above technique is not suitable for real-time broadcasting by multicast, which requires real-time, because retransmission is performed when data is lost. Accordingly, an object of the present invention is to solve the above-mentioned problems, to distribute (broadcast) compressed digital data such as video and audio to a plurality of terminals through a network at the same time, and to decode and reproduce on the terminal side. In particular, it is an object of the present invention to provide a technology for protecting a data delay or loss caused by an overload on a network or a client terminal from being disturbed.

[0006]

According to the present invention, there is provided a multicast system in which a transmitting apparatus and a receiving apparatus communicate via a network, wherein the transmitting apparatus converts data to be transmitted into packets. Means for adding packet continuity information to each packet, and the receiving device determines the recoverability of predetermined information composed of packets from the continuity information added to the packet, and A determination means for outputting non-recoverable information in case of
Means for receiving, when receiving the unrecoverable information, a packet used for recovering the predetermined information from among the received packets.

Further, when the determining means determines that the minimum unit packet to be decoded by the decoding means has not been received based on the continuity information of the packet, it outputs non-restorable information. And Also,
When the information to be restored is a video, and it is determined that one frame cannot be formed by the received packet based on the continuity information of the packet, the information to be restored is output. I do.

When the information to be restored is voice and it is determined that one packet cannot be formed by the received packet based on the continuity information of the packet, the determination means outputs non-restorable information. It is characterized by doing. Further, the transmission device includes an encoding unit that converts video and audio into digital data.

[0009] Further, the receiving apparatus determines that there is no continuity from the continuity information added to the packet,
In addition, when the lost packet can be replaced, a replacement means for replacing the lost packet is provided. Further, when the packet to be replaced is a video, the replacement means restores the video by replacing the frame.

Further, when the packet to be replaced is voice, the replacing means replaces the data with data immediately before the replacement. The receiving device detects an overflow of packet reception and outputs overflow information. When receiving the overflow information, the receiving device includes, among the received packets, packets that constitute information that cannot be restored due to the overflow. And means for deleting.

The receiving device may detect underflow of packet reception and output underflow information. The receiving device may be configured to receive the underflow information and recover predetermined information from the received packet. Has a means for deleting the received packet and a stop means for temporarily stopping the decoding means.

Further, when the information to be decoded by the decoding means is a video, the stopping means continuously decodes a frame currently displayed and stops as a still image. . Further, the stopping means has means for stopping decoding when the information decoded by the decoding means is audio.

In a multicast system, a receiving apparatus converts data transmitted by a transmitting apparatus into packets, and receives packets obtained by adding continuity information of each packet to each packet. From the continuity information, it is determined whether or not predetermined information composed of packets can be restored, and when restoration is not possible, a determination unit that outputs unrecoverable information, Means for discarding a packet used for restoring the predetermined information among the packets.

In a multicast system, a transmitting device converts data into a packet and transmits the packet, and receives the packet. The receiving device detects an overflow of packet reception and outputs overflow information. When receiving the overflow information,
Means for deleting, from the received packets, packets constituting information for which information cannot be restored due to overflow.

In a multicast system, a transmitting device converts a data into a packet and transmits the packet, and receives the packet. The receiving device detects an underflow of packet reception and outputs underflow information. Means for receiving, when receiving the underflow information, and when it is impossible to restore the predetermined information in the received packet, means for deleting the received packet, stop means for temporarily stopping the decoding means, It is characterized by having.

Further, when the information to be decoded by the decoding means is a video, the stopping means continuously decodes a frame currently displayed and stops as a still image. . Further, the stopping means has means for stopping decoding when the information decoded by the decoding means is audio.

By means of these means, only the information which can be restored is reproduced, thereby preventing reproduction from being disturbed. A method of controlling a multicast in a transmission method and a reception method of a multicast system communicating over a network, comprising: converting data to be transmitted into packets, adding packet continuity information to each packet; A first determining step of determining the presence or absence of continuity based on the continuity information of the packet, and, by the first determining step, a step of restoring if it is determined that there is continuity,
According to the first determination step, when it is determined that there is no continuity, a second determination step of determining the possibility of restoring predetermined information composed of packets, and by the second determination step,
If the restoration is not possible, the method includes a step of restoring the packet and a step of discarding a packet used for restoring the predetermined information among the received packets if the restoration is impossible. It is characterized by the following.

Further, in the second determining step, it is determined whether or not a minimum unit packet for restoring predetermined information is received based on the continuity information of the packet. 19. The method for controlling a multicast system according to 18. In the second determining step, the information to be restored is video, and it is determined whether or not a packet for restoring one frame has been received based on the continuity information of the packet. .

[0019] In the second determining step, it is determined whether or not the information to be restored is voice and a packet for restoring one sound is received based on the continuity information of the packet. Features. By these steps, by reproducing only the information that can be restored, the disorder of the reproduction can be prevented.

[Detailed Description of the Invention]

[0021]

The present invention relates to a multicast system in which a transmitting device and a receiving device communicate via a network.

[0022]

Embodiments of the present invention will be described.
FIG. 1 is a configuration diagram of the present embodiment. In FIG. 1, reference numeral 10 denotes a server. The server 10 has an encoder unit 11 and a packet header processing unit 12, which will be described later. Then, the server 10 encodes the video and the audio in real time,
It distributes (broadcasts) simultaneously to a plurality of client terminals 30 via a network 20 described later.

Reference numeral 11 denotes an encoder unit. The encoder unit 11 encodes video and audio in real time,
It converts to digital data. Reference numeral 12 denotes a packet header processing unit. The packet header processing unit 12 packetizes the data encoded by the encoder unit 11, and adds cyclic data indicating the order of the packets to the header of the packet.

Reference numeral 20 denotes a network. In the present embodiment, the server 10 and a client terminal 30 described later are connected via the network 20. Reference numeral 30 denotes a client terminal. The client terminal 30 includes a decoder unit 31, a continuity determination processing unit 32, a packet determination processing unit 33, a data discard processing unit 34, a data replacement processing unit 35, a data buffer unit 36, and a buffer amount management unit 3, which will be described later.
7 and a decoder control processing unit 38. The client terminal 30 receives a packet distributed (broadcast) from the server 10 via the network 20, decodes the data of the packet, and displays a video and reproduces a sound.

Reference numeral 31 denotes a decoder unit. Decoder section 31
Is to decode the data and reproduce video and audio. 32 is a continuity determination processing unit. The continuity determination processing unit 32 sends the network
, And determines whether or not the continuity of the packet is maintained based on the cyclic data of the packet. And a packet,
The identification result and the packet discrimination processing unit 33 described below are transferred.

Reference numeral 33 denotes a packet discrimination processing unit. If the packet has continuity based on the identification result, the packet discrimination processing unit 33 performs minimum decoding (the decoding unit 31).
Is transferred to the data buffer unit 36 described later for each (the minimum data amount that can be decoded). If the packet has no continuity based on the identification result, the packet determination processing unit 33 determines the configuration of the lost packet, and determines whether data replacement is possible or not according to the data type of the packet and the degree of loss. Identify. As a result of the identification, if the data cannot be replaced, an instruction is given to a buffer amount management unit 37 and a decoder control processing unit 38 (indicating that data has been lost), which will be described later, and the decoder 37 does not reproduce incorrect data. Control. If the result of the identification indicates that the data can be replaced, the data of the packet is transferred to a data discarding unit 34 described later.

Reference numeral 34 denotes a data discard processing unit. The data discarding unit 34 discards halfway data of the video data so as to be a frame unit which is a minimum unit of display. Then, the data of the frame unit is transferred to the data replacement processing unit 35 described later. Similarly, the data is discarded so as to be in the minimum decoding unit for voice. Then, the data of the minimum decoding unit is transferred to the data replacement processing unit 35 described later.

Reference numeral 35 denotes a data replacement processing unit. In the case of video data, the data replacement processing unit 35 replaces the discarded frame with a similar frame without a difference,
The frame data is transferred to the data buffer unit 36.
In the case of voice, the data is replaced with the same data as the previous data. 36 is a data buffer unit. The data buffer unit 36 stores the data transferred from the packet discrimination processing unit 33 and the data replacement processing unit 35. Then, the data is extracted by a decoder unit 31 described later at regular intervals.

Reference numeral 37 denotes a buffer amount management unit. The buffer amount management unit 37 monitors the data amount of the data buffer unit 36. For example, if packet continuity can be ensured, but data transmission is delayed due to overload of the network 20, the data buffer unit 36 becomes empty,
Supply of reproduction data to the decoder unit 31 described later becomes insufficient. For this reason, the decoder unit 31 cannot perform perfect decoding, and video and audio disturbances occur. Therefore, the buffer amount management unit 37 monitors the data buffer unit 36, and issues an instruction to a later-described decoder control processing unit 38 when the data falls below the unit of reproduction, and the decoder unit 3
1 is controlled to suspend reproduction, wait for a state in which reproduction is possible, and resume decoding. When the data buffer unit 36 overflows due to a load on the terminal side or the like, the buffer amount management unit 37 causes the data stored in the data buffer unit 36 to be discarded. That is, the video and the audio are skipped, and the data is not disturbed by the overflow.

Reference numeral 38 denotes a decoder control processing unit. When a delay occurs in data transmission due to an overload of the network 20 or the like, the decoder control processing unit 38 controls the decoder unit 31 to suspend reproduction, waits for a reproducible state, and instructs to resume decoding. Next, the operation of the present embodiment will be described. First, the server 1 causes the encoder unit 11 to encode video and audio data. And the encoder unit 11
Transfers the encoded video and audio data to the packet header processing unit 12.

Next, the packet header processing unit 12 packetizes the encoded video and audio data. At this time, the packet header processing unit 12 adds cyclic data indicating the order for each packet. Then, the packet header processing unit 12 distributes (broadcasts) the packetized packet data to the client terminal 30 via the network 20.

The client terminal 30 is connected to the network 2
0, and transfers the packet data to the continuity determination processing unit 32. The continuity determination processing unit 32 transmits the cyclic data of the transferred packet data (hereinafter, referred to as packet data A) and the cyclic data of the packet data transferred immediately before the packet data A (hereinafter, referred to as packet data B). Compare with If there is no loss in the packet data, the cyclic data changes sequentially for each packet data. Therefore, when a loss occurs in the packet data, the value of the cyclic data is not continuous, and the interrupted packet data is sent to the client terminal 30. The continuity determination processing unit 32 determines whether there is continuity between the packet data A and the packet data B based on the comparison.
Next, the continuity determination processing unit 32 transfers the packet data A and the result of the continuity to the packet determination processing unit 33.

When it is determined that the received packet data has continuity,
It is configured in video and audio decoding units and transferred to the data buffer unit 36. The data stored in the data buffer unit 36 is sequentially transferred to the decoder unit 31 to perform a decoding process, and display of video and reproduction of audio are performed.

When it is determined that the received packet data does not have continuity, the packet determination processing unit 33 determines whether the packet data can be restored as described later. Judgment as to whether restoration is possible is based on the fact that the loss of packet data indicates that the B frame (Bidirectionally Predicti
ve-coded Picture), P frame (Predictive-codedPi)
cture) or in the case of an audio decoding unit, it is determined that restoration is possible. When the loss of the packet data is an I frame (Intra coded Picture) of the video,
Alternatively, when a plurality of packet data losses occur, it is determined that restoration is impossible.

The packet discriminating unit 33 transfers the packet data judged to be recoverable to the data discarding unit 34. The data discard processing unit 34 identifies the packet data by a reproduction unit (frame), and when a part of the reproduction unit (frame) following the lost packet data (temporal connection of video and audio) is lost. Discards the playback unit (frame). That is, in the case of a B frame, only the B frame is discarded, and in the case of a P frame, data up to the next I frame is discarded. Then, the remaining packet data is transferred to the data replacement processing unit 35.

The data replacement processing unit 35 replaces the discarded frame with a similar frame without a difference,
The data is transferred to the data buffer unit 36. For voice, the same data as the previous data is replaced. When it is determined that the data cannot be restored, the packet discrimination processing unit 33 instructs the buffer amount management unit 37 to discard the data including the data stored in the data buffer unit 36, and starts from the beginning of the reproducible data. Decoding process.

At this time, the decoder control processing section 38 is instructed so that the screen is not distorted and there is no abnormal sound in the sound, and a stop process (still image) in a frame before the display is interrupted and a sound mute process are performed. As a result, the decoder unit 31 can prevent the reproduction from being disrupted until decodable data is input.
Next, an explanation will be given of the operation of the present embodiment when the video and audio data overflow and underflow due to the delay in the arrival of the video and audio data and the processing (or resources) of the terminal.

Overflow of video and audio data,
The underflow is determined by monitoring the state of the data buffer unit 36 by the buffer amount management unit 37 of the client terminal 30. That is, the buffer amount management unit 37 looks at the capacity of the data buffer unit 36, determines that the data is smaller than the decoding unit, determines that there is an underflow, instructs the decoder control processing unit 38, and displays the screen until the playback is determined to be possible. To stop (still image) and mute the sound.

When the data buffer unit 36 exceeds a certain capacity, the buffer amount management unit 37 determines that an overflow has occurred, discards the data stored in the data buffer unit 36, and starts from the beginning of the reproducible data. This is a decoding process. Further, data to be transmitted by thinning out frames in order to reduce the amount of data in the network is transferred to the data discarding unit 34 by the packet discriminating unit 33. Then, the data discarding unit 34 converts the data into
The data is transferred to the data replacement processing unit 35.

The data replacement processing unit 35 replaces the data with a similar frame without a difference for the thinned-out frame, and transfers the data to the data buffer unit 36. Next, another embodiment will be described. FIG. 2 is a configuration diagram of another embodiment. In the description of the other embodiments, the description of the same parts as those of the present embodiment will be omitted.
Only different parts will be described.

In FIG. 2, reference numeral 40 denotes a client terminal.
The client terminal 40 includes a decoder unit 31, a data buffer unit 36, a buffer amount management unit 37, a decoder control processing unit 38, a continuity determination processing unit 42 described later, and a data discard processing unit 43 described later. Client terminal 4
Numeral 0 is for receiving data distributed (broadcast) from the server 10 via the network 20, decoding the data, and displaying video and reproducing audio.

Reference numeral 42 denotes a continuity determination processing unit. The continuity determination processing unit 41 receives packet data distributed (broadcast) from the server 10 via the network 20, and based on information indicating continuity added to the head of the packet data, It is determined whether continuity is maintained. When the continuity of the packet data is maintained, the packet data is transferred to the data buffer unit 36 for each minimum decoding (the minimum data amount that can be decoded by the decoding unit 31). If the continuity of the packet data is not maintained, the packet data is transferred to the data discard processing unit 43.

Reference numeral 43 denotes a data discard processing unit. The data discarding unit 43 discards halfway data of the video data so as to be a frame unit which is a minimum unit of display. Then, the data in frame units is transferred to the data buffer unit 36. Similarly, the data is discarded so as to be in the minimum decoding unit unit, and the data is transferred to the data replacement process 34.

Next, the operation of another embodiment will be described. The normal operation, the delay of data by the network 20, and the failure by the client terminal 30 are the same as those in the present embodiment, and the description is omitted. Here, the loss of packet data due to a load on the network 20 will be described.

First, when a loss occurs in a packet transmitted from the network 20, the value of the cyclic data indicating the packet number added to each packet does not continue,
The interrupted packet is sent to the client terminal 40.
This packet is determined by the continuity determination processing unit 42 to have no continuity. Then, the continuity determination processing unit 42 transfers the packet and information indicating that the packet has no continuity to the data discard processing unit 43.

The data discard processing unit 43 having received the transfer
Instructs the buffer amount management unit 37 to discard data including the data stored in the data buffer unit 36, and performs decoding processing from the beginning of reproducible data. At this time, the decoder control processing unit 38 performs display stop processing (still image) and audio mute processing in a frame before the display is interrupted so that the screen is not distorted and the sound is not abnormal. As a result, the decoder unit 31 can prevent the reproduction from being disrupted until decodable data is input.

That is, since it is difficult to completely restore the missing data, the restoring process is omitted, and the configuration is simplified by performing processing that does not disturb the video.

[0048]

The first effect is that display and reproduction can be performed without disturbing video and audio due to a data transmission failure in a network that may occur by multicasting. The reason is that a means for analyzing the transfer packet and the data structure on the terminal side to determine underflow / overflow causing a problem and controlling decoding processing is provided.

The second effect is that the reproduction can be performed without disturbing the video and the audio even when the load on the terminal side or the failure due to the inconsistency of the control time with the server side is disturbed. The reason is that decoding processing can be predicted by providing a process for constantly monitoring the data buffer held on the terminal side. The third effect is that the transmission device can also perform interpolation on data subjected to frame thinning without interfering with reproduction. The reason is that a means for replacing frame data is provided on the terminal side.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of the present embodiment according to the present invention.

FIG. 2 is a diagram showing a configuration of another embodiment according to the present invention.

[Explanation of symbols]

 Reference Signs List 10 server 11 encoder unit 12 packet header processing unit 20 network 30 client terminal 31 decoder unit 32 continuity determination processing unit 33 packet determination processing unit 34 data discard processing unit 35 data replacement processing unit 36 data buffer unit 37 buffer amount management unit 38 decoder Control processing unit

Claims (21)

[Claims] 1. A multicast system in which a transmitting device and a receiving device communicate via a network, wherein the transmitting device converts data to be transmitted into packets and adds packet continuity information to each packet. Determining means for determining, based on the continuity information added to the packet, the recoverability of predetermined information composed of a packet, and outputting the non-recoverable information when recovery is not possible; Means for receiving the non-restorable information, and for discarding a packet used for restoring the predetermined information from among the received packets, a means for discarding the packet. 2. The method according to claim 1, wherein the determining unit outputs, based on the continuity information of the packet, non-recoverable information when it is determined that the minimum unit packet to be decoded by the decoding unit has not been received. The multicast system according to claim 1, wherein 3. When the information to be restored is a video, and it is determined that one frame cannot be formed by the received packet based on the continuity information of the packet, the determination unit outputs unrecoverable information. The multicast system according to claim 1, wherein the multicasting is performed. 4. If the information to be restored is audio and it is determined that one tone cannot be formed by a received packet based on the continuity information of the packet, the restoration unit outputs non-restorable information. 2. The method according to claim 1, wherein
The multicast system according to 1. 5. The multicast system according to claim 1, wherein the transmission device has an encoding unit that converts video and audio into digital data. 6. The receiving device according to claim 1, further comprising a replacement unit that replaces the lost packet when it is determined that there is no continuity from the continuity information added to the packet and when the lost packet can be replaced. The multicast system according to any one of claims 1 to 5, characterized in that: 7. The multicast system according to claim 1, wherein said replacing means restores the video by replacing frames when the packet to be replaced is a video. 8. The multicast system according to claim 1, wherein said replacement means replaces the packet to be replaced with data immediately before replacement when the packet is voice. 9. The receiving device detects an overflow of packet reception, and outputs overflow information. Upon receiving the overflow information, the receiving device extracts information of the received packet from which information cannot be restored due to the overflow. The multicast system according to any one of claims 1 to 8, further comprising: means for deleting constituent packets. 10. The receiving apparatus detects a packet receiving underflow and outputs underflow information, and the receiving apparatus receives the underflow information and recovers predetermined information from the received packet. 10. The multicast system according to claim 1, further comprising: means for deleting the received packet; and stopping means for temporarily stopping decoding means. 11. The information processing apparatus according to claim 1, wherein said stopping means has a means for, when the information decoded by said decoding means is a video, continuously decoding a frame currently displayed and stopping as a still image. Claim 1 to Claim 1
0. The multicast system according to any one of 0. 12. The apparatus according to claim 1, wherein said stopping means has means for stopping decoding when information decoded by said decoding means is audio.
The multicast system according to any one of the above. 13. A receiving apparatus for converting data transmitted by a transmitting apparatus into packets in a multicast system and receiving packets obtained by adding packet continuity information to each packet, wherein the receiving apparatus is configured to add a packet to a packet. From the continuity information, it is determined whether the predetermined information composed of packets can be restored, and if restoration is not possible, a determination unit that outputs unrecoverable information; Means for discarding a packet used for restoring the predetermined information from among the packets. 14. In a multicast system, a transmitting device converts a data into a packet, transmits the packet, and receives the packet. The receiving device detects an overflow of packet reception and outputs overflow information. Receiving means for receiving the overflow information, and for deleting packets constituting information that cannot be restored due to the overflow among the received packets. 15. In a multicast system, a transmitting device converts data into a packet and transmits the packet, and receives the packet. The receiving device detects an underflow of packet reception and outputs underflow information. Means for receiving the underflow information, and when it is not possible to restore predetermined information in the received packet, means for deleting the received packet; stopping means for temporarily stopping decoding means; The receiving device according to claim 14, comprising: 16. The image processing apparatus according to claim 16, wherein the stopping unit has a unit that, when the information to be decoded by the decoding unit is a video, continuously decodes a currently displayed frame and stops as a still image. The receiving device according to claim 14. 17. The apparatus according to claim 14, wherein said stopping means has means for stopping decoding when information decoded by said decoding means is audio.
7. The receiving device according to any one of 6. 18. A method of controlling a multicast in a transmission method and a reception method of a multicast system communicating via a network, wherein the step of converting data to be transmitted into packets and adding packet continuity information to each packet. A first determining step of determining the presence or absence of continuity based on the received continuity information of the packet; and a step of restoring if it is determined that there is continuity by the first determining step; In the first determining step, when it is determined that there is no continuity, a second determining step of determining the retrievability of predetermined information composed of packets, and in the case of being recoverable by the second determining step A packet used for restoring the predetermined information among received packets when restoration is not possible by the restoring step and the second determining step. Control method for a multicast system, comprising discarding step. 19. The method according to claim 20, wherein the second determining step determines whether a minimum unit packet for restoring predetermined information is received based on the continuity information of the packet. 19. The method for controlling a multicast system according to 18. 20. The second judging step judges whether or not the information to be restored is a video and a packet for restoring one frame has been received based on the continuity information of the packet. The method for controlling a multicast system according to claim 18, wherein: 21. The second judging step judges whether or not the information to be restored is voice and whether or not a packet for restoring one sound has been received based on the continuity information of the packet. The method for controlling a multicast system according to claim 18, wherein: