JP2006333367A - Video transmission system and video transmission method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a video transmission system for recovering, in a short time, the deterioration of regenerative video quality accompanied by packet omission without transmitting a retransmission packet if the packet omission occurs. <P>SOLUTION: When the packet omission is detected in a video receiving section 20 in the video transmission system including a video transmitting section 10 and the video receiving section 20, the video transmitting section 10 performs partial intra-encoding on partial video data corresponding to a reproducing position of an omitted packet and transmits the video data in a frame to succeedingly be transmitted. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention generally relates to a video transmission system that packetizes and transmits video encoded data, and more particularly, to a recovery technique for video quality degradation caused by packet loss.

  In recent years, video transmission systems in which video encoded data obtained by digitally encoding video signals (hereinafter simply referred to as encoding) are packetized and transmitted over a network have become common. In addition to digital television broadcasting, such a system is being applied to a video surveillance system that performs video surveillance using a surveillance camera installed in a remote place, for example.

  In such a video monitoring system, if a packet loss occurs during transmission, the playback video quality deteriorates when the video signal is played back on the receiving side. Normally, in order to recover the degradation of the reproduced video quality due to the occurrence of packet loss, intra-frame encoded data (I block or I picture in the MPEG system) called a so-called intra encoded frame is required. However, due to the recent pursuit of coding efficiency, the transmission interval of intra-coded frames, which is a recovery point for deterioration, has increased, and it takes time to recover.

Thus, conventionally, a system has been proposed in which when a packet is lost, a request for retransmission of the lost packet is made from the reception side to the transmission side (see, for example, Patent Documents 1 and 2).
JP 11-331839 A JP 2000-341252 A

  By retransmitting the missing packet by the conventional system as described above, it is possible to recover the degradation of the reproduced video quality. However, in the conventional system, the video display is stopped for the waiting time until the retransmission packet arrives. For this reason, on the receiving side, the transmitted encoded data must be buffered to some extent, so that a delay occurs in video reproduction. In addition, by transmitting a retransmission packet, it consumes network bandwidth and causes further packet loss.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a video transmission system that realizes, in a short time, the reproduction video quality degradation due to packet loss without transmitting retransmission packets when packet loss occurs. It is in.

  According to an aspect of the present invention, when a packet loss is detected, a partial video of a frame to be transmitted next corresponds to a reproduction position (position or range on the screen) of a frame of encoded video data included in the corresponding packet. This is a video transmission system that transmits data by intra-frame coding (partial intra coding). The reproduction position corresponds to the display position or display range on the display screen when it is assumed that the video encoded data of the missing packet has been reproduced.

  A video transmission system according to an aspect of the present invention is a video transmission system in which encoded data obtained by encoding a video frame is divided and transmitted in packet units. Determining means for determining whether the encoded data included in the packet in which the loss is detected by the means is encoded data of a reference frame used as a predicted image in decoding of other encoded data; In the case where the determination result of the determination means is encoded data of a reference frame, a means for specifying a reproduction position in the video frame of the encoded data included in the packet in which the loss is detected, and a transmission target video frame On the other hand, there is provided an encoding means for encoding the partial video data corresponding to the reproduction position into intra-frame encoded data.

  According to the present invention, it is possible to provide a video transmission system that realizes, in a short time, recovery of degradation in reproduction video quality due to packet loss without transmitting retransmission packets when packet loss occurs. .

  Embodiments of the present invention will be described below with reference to the drawings.

[System configuration]
FIG. 1 is a block diagram illustrating a configuration of a video transmission system according to the present embodiment.

  As shown in FIG. 1, the system is roughly divided into a video transmission unit 10, a video reception unit 20, and a network 30 that is a transmission medium.

  The video transmission unit 10 includes a video encoding unit 100 that encodes a video signal 110 including a plurality of video frames input from the video input device 11, a video packet transmission unit 101, a packet information table 102, and packet loss information. The receiving unit 103 and the comparison error position detecting unit 104 are included. The video frame is simply referred to as a frame.

  The video input device 11 is a video shooting camera such as a surveillance camera. The video encoding unit 100 encodes the video signal 110 in units of frames in accordance with a predetermined standard, for example, the Visual (ISO / IEC14496-2) standard in MPEG4.

  The video packet transmission unit 101 converts the video encoded data (hereinafter simply referred to as encoded data) 111 encoded by the video encoding unit 100 in units of frames into a transmission signal in units of packets, and receives video through the network 30. To the unit 20. The video packet transmission unit 101 uses UDP (User Datagram Protocol) that does not cause retransmission of data due to an error as a transport layer protocol for video packet transmission.

  The video encoding unit 100 outputs the encoded data 111 to the video packet transmission unit 101 and records packet information described later in the packet information table 102. Specifically, as shown in FIG. 2, the packet information table 102 records packet information related to an encoding mode and a position on the screen, which will be described later, in addition to the sequence number and time stamp included in the header of the transmitted packet. It is a table. In the packet information table 102, old recording data that has become unnecessary is sequentially overwritten and erased. The packet information table 102 has a table size determined by the network transmission delay and the processing time of the comparison error position detection unit 104.

  Here, the information regarding the position on the screen (the position on the screen) is information indicating a reproduction position that means a position or a range on the display screen when the video reception unit 20 reproduces the video encoded data. . Further, in the video encoding unit 100, video encoding is performed in units of blocks obtained by dividing the screen into squares, and is transmitted in order from the upper left position on the screen to the lower right position. For this reason, in the packet information table 102, the position on the screen consisting of the start block and the end block for each packet is recorded.

  Note that packetization is performed in accordance with RFC (Request for Comments) 3016 of IETF (Internet Engineering Task Force), for example. For this reason, encoded data of a plurality of frames (screens) is not included in the same packet. In other words, each packet consists only of partially encoded data in one frame.

  On the other hand, the video reception unit 20 includes a video packet reception unit 200, a video decoding unit 201, a packet loss detection unit 202, and a packet loss information transmission unit 203.

  The video packet receiving unit 200 receives the packet transmitted from the video transmitting unit 10, extracts a payload (encoded data that is video content) from the packet, and outputs the payload to the video decoding unit 201. The video decoding unit 201 decodes the encoded data in accordance with, for example, the Visual (ISO / IEC14496-2) standard in MPEG4, and outputs the decoded data to the video output device 21. The video output device 21 includes a video monitor that reproduces the decoded video signal and displays it on the display screen.

  The video packet receiver 200 transfers the received packet header to the packet loss detector 202. As shown in FIG. 3, the packet header has a format defined by, for example, RFC (Request for Comments) 3550 standard of Internet Engineering Task Force (IETF).

  Here, sequence numbers are consecutively assigned for each packet. The time stamp stores time information of the frame. Packets having the same time stamp are generated by dividing the same frame. Specifically, as shown in FIG. 2, for example, packets with sequence numbers 0123 and 0124 each have encoded data having different positions on the screen in the same frame.

  The packet loss detection unit 202 extracts a sequence number from the transferred header, and determines that a packet is missing when the sequence number is discontinuous. Specifically, for example, when the header of sequence number 0125 is input next to sequence number 0123, packet loss detection section 202 determines that the packet having the header of sequence number 0124 has been lost.

  When the packet loss detection unit 202 detects a packet loss, the packet loss detection unit 202 outputs the detection result to the packet loss information transmission unit 203. The packet loss information transmission unit 203 transmits information corresponding to the detection result to the packet loss information reception unit 103 of the video transmission unit 10 through the network 30. The packet loss information transmission unit 203 transmits, as transmission information, packet loss information including the sequence numbers and time stamps of the previous and subsequent lost packets.

  In the video transmission unit 10, the packet loss information reception unit 103 transfers the received packet loss information to the comparison error position detection unit 104. The comparison error position detection unit 104 compares the packet information recorded in the packet information table 102 at the time of transmission with the received packet loss information, and the encoded data included in the packet lost due to the error is reproduced. Specify the position on the screen (the playback range on the screen). The comparison error position detection unit 104 outputs error position information indicating the specified on-screen position to the video encoding unit 100.

  Here, the comparison error position detection unit 104 is encoded data of a reference frame used (referenced) as a predicted image in decoding of another frame (encoded data) as an encoding mode of a lost packet. In the case of a P frame (P picture) and an I frame (I picture), error position information is output to the video encoding unit 100. That is, the comparison error position detection unit 104 does not output error position information when the encoding mode of the lost packet is B frame.

  A B frame is encoded data that is not referred to as a predicted image in decoding of another frame. The I frame is intra-frame encoded data that can be decoded without referring to another frame as a predicted image. The I frame and the P frame are referred to as predicted images when other frames are decoded.

  As will be described later, the video encoding unit 100 performs intra-frame encoding (hereinafter referred to as partial intra-frame coding) of a portion (position on the screen) specified by the error position information from the comparison error position detection unit 104 in the P frame to be transmitted next. May be referred to as encoding). In addition, since motion compensation is performed, it is preferable that the peripheral portion that is the motion vector range is also a target of partial intra coding.

[Function and effect]
Hereinafter, with reference to FIG. 4 to FIG. 6, the operational effects relating to the system of the present embodiment will be described.

  First, in the video transmission unit 10, the video packet transmission unit 101 receives a packet including the encoded data 111 obtained by encoding the video signal 110 input from the video input device 11, which is a monitoring camera, through the network 30. To the unit 20.

  The video receiver 20 decodes the encoded data included in the received packet into a video signal and outputs it to the video output device 21. The video output device 21 reproduces the video signal decoded in units of frames and displays it on the display screen of the video monitor.

  Here, as shown in the flowchart of FIG. 5, in the video reception unit 20, when the packet loss detection unit 202 detects a missing packet from a series of received packets, the packet loss information transmission unit 203 displays the packet loss information as video. It transmits to the packet loss information receiving part 103 of the transmission part 10 (YES of step S1).

  When the comparison error position detection unit 104 receives the packet loss information from the packet loss information reception unit 103, the comparison error position detection unit 104 compares the packet loss information with the packet information recorded in the packet information table 102 at the time of transmission. Identify the packet.

  The comparison error position detection unit 104 determines whether or not the encoded data is a B frame based on the encoding mode of the packet lost due to the error (step S3). When the determination result is a B frame, the comparison error position detection unit 104 is encoded data that is not referred to as a predicted image in the decoding of another frame. (Partial intra coding) is not executed (YES in step S3).

  On the other hand, when the encoding mode of the missing packet is a P frame or I frame other than the B frame, the comparison error position detection unit 104 displays the position on the screen of the packet missing from the packet information table 102 (the reproduction range on the screen). ) Is specified (step S4).

  The comparison error position detection unit 104 outputs error position information indicating the specified on-screen position to the video encoding unit 100, and requests partial intra encoding for the next frame (step S5).

  Next, the operation of the video encoding unit 100 of the video transmission unit 10 will be described with reference to the flowchart of FIG.

  When receiving the error position information and the partial intra coding request from the comparison error position detecting unit 104, the video coding unit 100 determines the coding mode (coding type) of the frame to be transmitted next (Step S11). YES, S12). Here, when there is no request for partial intra coding from the comparison error position detection unit 104, the video coding unit 100 executes normal coding (step S13). That is, the video encoding unit 100 executes encoding according to a specified encoding mode (encoding type).

  In addition, when the video encoding unit 100 receives a request for partial intra encoding and the encoding mode of a frame to be transmitted next is a B frame, the video encoding unit 100 is not referred to as a predicted image in decoding of another frame. Therefore, normal encoding is executed (YES in step S12, S13). That is, the error recovery process (partial intra coding in the next frame) in this embodiment is not executed.

  On the other hand, when the encoding mode of the frame to be transmitted next is other than the B frame, the video encoding unit 100 clears the request for partial intra encoding, and then the encoding mode of the frame to be transmitted next is It is determined whether it is a P frame (NO in step S12, S14, S15).

  When the encoding mode of the frame to be transmitted next is I frame, the video encoding unit 100 performs intra encoding on the entire screen. As a result, the video encoding unit 100 ignores the request for partial intra encoding and performs normal encoding. Is executed (NO in step S15, S13).

  In addition, when the encoding mode of the frame to be transmitted next is P frame, the video encoding unit 100 executes partial intra encoding because it is referred to as a predicted image in decoding of other frames. (YES in step S15, S16). That is, the video encoding unit 100 performs intra-frame encoding on the partial video data corresponding to the position on the screen specified by the error position information from the comparison error position detection unit 104 in the P frame to be transmitted next. To do. In this case, the video encoding unit 100 may perform intra-frame encoding on the partial video data corresponding to the designated on-screen position and its peripheral part.

  As described above, in the system of this embodiment, when a packet loss is detected, if the encoding mode of the packet is particularly P frame, the playback position on the screen, that is, the position on the screen (the position on the screen). ). Then, when the frame to be transmitted next is a P frame, the video encoding unit 100 performs partial intra encoding that intra-codes the partial video data corresponding to the specified position on the screen.

  Therefore, the system according to the present embodiment does not transmit a retransmission packet when a packet loss (packet loss) occurs, and partially intra-codes a partial video corresponding to a lost packet in a P frame to be transmitted thereafter. By transmitting, it is possible to shorten the time taken for the quality degradation of the reproduced video to recover. Hereinafter, a specific description will be given with reference to FIG.

  If, for example, two packets are lost from a P frame being transmitted, as shown in FIG. 4, in the decoding of the B frame that refers to the P frame as a predicted image, the quality of the reproduced video deteriorates. To do. Here, in the system of the present embodiment, error recovery is realized by performing partial intra-coding and transmitting a partial video corresponding to a missing packet in a P frame to be transmitted thereafter. As a result, in the decoding of the B frame that refers to the P frame that has been recovered from the error as the predicted image, the quality of the reproduced video is recovered.

  In the system of this embodiment, the video display is not stopped for the waiting time until the retransmission packet arrives unlike the conventional system, and there is no delay in video playback. Further, the receiving side can reduce the buffering amount for buffering the transmitted encoded data to some extent, and can suppress the consumption of the network bandwidth to the minimum.

  In addition, since there is no reference from other frames at the time of decoding, partial intra coding is not executed in the case of a B frame in which further deterioration of video quality is not propagated or an I frame for intra-frame coding of the entire screen. As a result, a decrease in transmission efficiency can be minimized.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

The block diagram which shows the structure of the video transmission system regarding embodiment of this invention. The figure which shows an example of the packet information table regarding this embodiment. The figure which shows an example of the header format of the packet regarding this embodiment. The figure for demonstrating the effect of this embodiment. The flowchart for demonstrating operation | movement of the system regarding this embodiment. The flowchart for demonstrating operation | movement of the system regarding this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video transmission part, 11 ... Video input device, 20 ... Video reception part, 21 ... Video output device,
30 ... Network, 100 ... Video encoder, 101 ... Video packet transmitter,
102 ... Packet information table, 103 ... Packet loss information receiver,
104: Comparison error position detection unit, 200: Video packet reception unit,
201 ... Video decoding unit, 202 ... Packet loss detection unit,
203: Packet loss information transmission unit.

Claims (10)

In a video transmission system in which encoded data obtained by encoding a video frame is divided into packets and transmitted,
Detection means for detecting packet loss during transmission;
Determining means for determining whether or not the encoded data included in the packet whose loss is detected by the detecting means is encoded data of a reference frame used as a predicted image in decoding of other encoded data When,
In the case where the determination result of the determination unit is the encoded data of the reference frame, the specifying unit that specifies the reproduction position in the video frame of the encoded data included in the packet in which the loss is detected;
A video transmission system comprising encoding means for encoding partial video data corresponding to the reproduction position into intra-frame encoded data for a video frame to be transmitted. A table means for recording information for identifying a packet to be transmitted, an encoding mode of encoded data included in the packet, and information indicating the reproduction position;
The determination unit is configured to determine whether the encoded data included in the packet in which the loss is detected is encoded data of a reference frame based on an encoding mode recorded in the table unit. The video transmission system according to claim 1, wherein: In the case where the video frame to be transmitted is a frame in which the entire screen is encoded into intra-frame encoded data, the encoding means does not perform intra-frame encoding of the partial video data,
The video transmission system according to claim 1, wherein the video transmission system is configured to perform intra-frame encoding of full-screen video data.   The encoding means performs encoding in a specified encoding mode when the video frame to be transmitted is a frame made of encoded data that is not used as a predicted image in decoding of other encoded data. The video transmission system according to claim 1, wherein the video transmission system is configured to be executed. Video receiving means for receiving the transmitted packet, decoding the encoded data and reproducing it on the display screen;
The video transmission system according to claim 1, wherein the detection unit is included in the video reception unit. Table means for recording information for identifying a packet to be transmitted, an encoding mode of encoded data included in the packet, and information indicating the reproduction position;
Means for generating information for identifying packets transmitted before and after the lost packet as packet loss information for identifying the lost packet when a packet loss is detected by the detection means;
The determination unit is configured to determine whether encoded data included in the packet in which the loss is detected is encoded data of a reference frame based on the table unit and the packet loss information. The video transmission system according to claim 1. Including the determination means and the identification means,
When the determination result of the determination means is encoded data of a reference frame, a request for intra-coding the partial video data corresponding to the reproduction position by the specifying means in the video frame to be transmitted next The video transmission system according to any one of claims 1 to 6, further comprising means for executing the encoding means. In a video transmission method applied to a video transmission system in which encoded data obtained by encoding a video frame is divided into packets and transmitted,
Detecting packet loss during transmission;
Determining whether the encoded data included in the packet in which the loss is detected is encoded data of a reference frame used as a predicted image in decoding of other encoded data;
When the determination result is encoded data of a reference frame, specifying a reproduction position in the video frame of encoded data included in the packet in which the loss is detected;
And a step of encoding partial video data corresponding to the reproduction position into intra-frame encoded data for a video frame to be transmitted.   When the video frame to be transmitted is a frame in which the entire screen is encoded into the intra-frame encoded data, the intra-frame encoding of the full-screen video data is performed without executing the intra-frame encoding of the partial video data. The video transmission method according to claim 8, further comprising the step of:   When the transmission target video frame is a frame made of encoded data that is not used as a predicted image in decoding of other encoded data, the method further includes a step of executing encoding in a specified encoding mode. The video transmission method according to claim 8.

Images