KR20130141368A - Reception device and program for reception device - Google Patents

Abstract

The present invention provides a reception device capable of being processed in real time with the scalable encoding characteristics even though the stream type data is transmitted by a TCP. The reception device comprises a frame detection unit (85) which detects the base layer reception completion time that a base layer of an object being replayed is equipped by a transmission image packet received in a reception unit (81) and a timer unit (86) which notifies the elapse of a timeout period to a reception completion processing unit (84) in case where the predetermined timeout period elapses from the time that the base layer of the object being replayed is equipped in the frame detection unit (85), wherein if the timer unit (86) notifies the elapse of the timeout period to the reception completion processing unit (84), a reception completion packet for the unreceived transmission image packet is transmitted to a transmission device (101). [Reference numerals] (100) Reception device;(101) Transmission device;(2) Encoding control unit;(3) Image encoding unit;(4) Image data memory unit;(5) Image characteristic specifying unit;(6) Priority extension layer setting unit;(7) Transmission processing unit;(8) Reception processing unit;(9) Image decoding unit;(AA) Network

Description

Reception device and program for reception device

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a receiver and a program for a receiver for receiving a transmission video packet transmitted from a transmitter in a data transmission system for transmitting video data as stream data.

For example, when data of video data is transmitted via Internet communication, stream type data transmission is used, in which a reproduction process is performed in parallel while transmitting data from a transmitting apparatus to a receiving apparatus, rather than a download type transmission system.

When such stream-type data transmission of video data is performed, encoded video data that is scalable-encoded video data is used so as to obtain a transmission video result corresponding to a line state or a reception terminal capability.

An example of scalable coding is SVC (Scalable Video Coding) within the H.264 video codec standard. The data structure of this codec can be divided into a base layer composed of the minimum necessary video elements as shown in Fig. 6, and an extension layer capable of improving image quality by combining with the base layer.

The base layer, which is a highly reliable channel, is a data stream that is essential when a video is reproduced on the receiving side, and is transmitted and received with high reliability by redundancy such as forward error correction (FEC).

On the other hand, the enhancement layer, which is a low reliability channel, is further divided into a spatial layer contributing to the high definition of the image, a temporal layer contributing to the improvement of the frame rate of the video, and an S / N ratio layer contributing to the improvement of the S / N ratio of the image. Divided.

For example, since the FEC process or the like has not been performed with respect to the extended layer, it is difficult to reach the receiving apparatus when the line condition worsens. Since the receiving device always receives the necessary base layer, it is possible to obtain a transmission video result suitable for the line quality by properly decoding the combination of the arrival layer from the transmitting device and the base layer.

However, in case of data transmission using scalable encoded coded image data, as shown in Patent Literature 1, the transmission protocol uses a real time protocol (RTP), which is a transmission method that does not perform retransmission, thereby realizing more reliable data transmission. Give your sex first.

In addition, Patent Document 1 discloses a data transmission system configured to perform retransmission processing for some layers while using RTP that has not been retransmitted in principle as a transmission protocol.

However, when performing stream type transmission such as moving pictures, TCP may be used as a transmission protocol to maintain reliability by performing retransmission processing. When the encoded video data scalable by TCP is transmitted, if there is a transmission video packet not received by the receiving apparatus, retransmission processing is performed on all transmission video packets that have not reached. Therefore, when the data encoded scalable by TCP is transmitted, in the worst case, if only the video is to be reproduced, the transmission video packet of the enhancement layer that does not need to be reached by the receiving apparatus is reliably retransmitted. Therefore, the quality at the time of moving picture reproduction can be fixed to the highest, but since the waiting time until all transmission video packets are received, the real-time property is impaired.

In other words, a simple combination of TCP and scalable coding cannot automatically obtain an image quality suitable for line quality, and thus cannot obtain a best effort type result like combining RTP and scalable coding. Therefore, it is not possible to exhibit the characteristics of the SVC, and as in the prior art, the bitrates should be encoded according to the line state, or a plurality of encoded video data should be prepared.

Moreover, the data transmission system of patent document 2 has a structure only for a receiving apparatus on the premise that the sending apparatus repeats a retransmission process, unless the receiving apparatus receives a reception completion packet with respect to the transmitted video packet. It is configured to ensure that the base layer is received by the receiving device and that the other extended layer cannot be received in a uniform manner.

More specifically, the reception apparatus does not transmit the reception completion packet to the transmission apparatus as usual for what was not received for the base layer and at the same time does not actually receive the reception of the extended layer. The retransmission process is forcibly interrupted for the enhancement layer by sending the completion packet to the transmitter.

However, in such a receiver, if there is any transmission video packet of the enhancement layer that the receiver could not receive, the retransmission process of the enhancement layer is interrupted, so that if the line condition worsens, the decoder cannot hardly decode the enhancement layer. In some cases, only the base layer is transmitted.

In other words, even if there is a request to play a video with minimum protection of the base layer and another priority extension layer in such a receiver, regardless of the line condition, the reception layer cannot protect the extension layer. The best effort service cannot be provided while reflecting the symbol of.

<Prior Art Literature>

Patent literature

Patent Document 1. Japanese Patent No. 3757857

Patent Document 2: Japanese Patent Application Laid-Open No. 9-307510

Accordingly, the present invention has been made in view of the above-described problems, and even in the case of stream type data transmission by TCP, the scalable coding characteristics are exhibited to enable best effort type video data transmission and secure real-time, An object of the present invention is to provide a receiver and a program for a receiver capable of protecting part of an enhancement layer. In other words, the present invention provides the best effort while protecting a part of the base layer and the enhancement layer only by the configuration of the receiving device, which is a client, even if the transmitting device such as a server has a retransmission function but does not have a configuration for setting which layer to retransmit. The purpose is to make it possible to receive a sentence service.

That is, the present invention is a receiver for receiving a transmission video packet transmitted from a transmission device in a data transmission system for transmitting video data in a stream type data, wherein the transmission device is encoded video data and encoded video including a base layer and an enhancement layer. An image data storage unit for storing layer data representing a layer of data, a transmission unit for generating a transmission image packet storing layer data corresponding to the encoded image data and transmitting the packet to the receiving apparatus, and It is assumed that a reception complete packet indicating a transmission video packet received at a reception device is received from the reception device, and that a retransmission processing unit for retransmitting a transmission video packet not receiving a reception completion packet from the reception device is provided to the transmission unit. Shall be.

And a reception completion processing unit for transmitting a reception completion packet indicating a transmission video packet received at the reception unit to the transmission apparatus, and a layer of the transmission image packet received at the reception unit based on the layer data. A frame detection unit for detecting a base layer reception completion time, which is a time when a base layer of a playback target frame is provided by a layer identification unit, a transmission video packet received by the reception unit, and a timeout time predetermined at the completion time of the base layer reception completion; And a timer unit for notifying the reception completion processing section that the timeout time has elapsed, when the elapsed time has elapsed. Receive completion packet to the sender It characterized in that it is configured to transmit.

If this is the case, the reception completion processing section is configured to send a reception completion packet to the transmission apparatus for the unreceived transmission video packet when the timer section has notified that the timeout time has elapsed, so that retransmission processing such as TCP is performed. Even in the case of performing data transmission using an excitation transmission protocol, even when the circuit condition is bad, only the base layer can be reliably received by the retransmission process, and the retransmission process can be stopped for the other extended layers.

The interruption of the retransmission processing of the enhancement layer is configured to occur after the timeout time elapses from the completion of the base layer reception, so that the unreceived enhancement layer can be protected by retransmission even at least until the timeout time elapses. .

That is, the reception apparatus of the present invention does not stop the retransmission process by the transmitting apparatus by uniformly transmitting the received completion packet to the unreceived enhancement layer, but a grace period called a timeout time is provided. It is also intended to protect.

On the other hand, if the line condition is good, all of the enhancement layers are received by the receiving apparatus, so that the best quality reproduction is possible.

Therefore, according to the present invention, even if there is no particular configuration in the transmitting apparatus, the configuration of the receiving apparatus enables the reproduction of video quality of image quality according to the line condition even in the transmission protocol with retransmission processing such as TCP, thereby providing the best effort type service. Can be. In addition, it can be a best effort type service that reflects the user's demand for the image quality of protecting not only the base layer but also a part of the extended layer.

In order to protect part of the enhancement layer in the same manner as the base layer while not impairing the real time of video playback, the timeout time may be configured to be terminated before the playback timing of the playback target frame.

In order to protect a part of the enhancement layer as the priority enhancement layer so that the priority enhancement layer is reliably received, a priority enhancement layer setting unit for setting a priority extension layer which is an extension layer to be retransmitted is further provided, and the frame detection unit further comprises: the base layer; It is configured to detect a priority completion layer reception completion time that is a reception completion time and a priority enhancement layer is equipped, the timeout time from the slower time point of the base layer reception completion time and the completion of the reception of the priority enhancement layer. When the elapsed time has elapsed, the reception completion processing unit may be configured to notify that the timeout time has elapsed. If this is the case, the base layer and the priority enhancement layer are protected by the retransmission process and are received by the receiving device, and at the same time, the retransmission process is stopped for the extension layer that is not received after the timeout time elapses, so that the video playback according to the line condition can be stopped. It becomes possible.

In order to be able to detect whether the base layer or the priority enhancement layer has been received only by the configuration of the receiver even without receiving special data from the transmitter, the frame detector uses the base image or the transmitted image packet received by the receiver. First, the temporary layer may be temporarily decoded, and the temporary decoding success point may be configured to be the base layer reception completion time or the first extended layer arrival time.

The reception completion processing unit may allow the transmission apparatus to stop the retransmission process for data other than the base layer and the priority enhancement layer with a simple configuration of the reception apparatus even if a detailed list of transmission video packets is not transmitted to the reception apparatus. Receiving the received complete packet for the unreceived transmission video packet having a sequence number smaller than the largest number among the sequence number of the received transmission video packet at the slower of the base layer reception completion time or the reception of the priority enhancement layer reception. It is sufficient to be configured to transmit to the transmitting device.

In order to add a function as a receiving device of the present invention to an existing receiving device so that a best effort type result can be obtained at the time of moving picture reproduction, at least a receiving completion packet indicating a transmission video packet received at the receiving unit is transmitted to the transmitting device. A base which is a point in time at which a reception completion processing unit, a layer identification unit for identifying a layer of a transmission video packet received by the reception unit based on layer data, and a base layer of a playback target frame are provided by the transmission image packet received at the reception unit; A frame detection unit for detecting a layer reception completion point and notifying the reception completion processing unit that the timeout time has elapsed when a predetermined timeout time has elapsed since the time at which the base layer of the frame to be reproduced is provided by the frame detection unit. And having a timer unit The reception completion processing unit is configured to transmit a reception completion packet to the transmission device for an unreceived transmission video packet when the reception completion processing unit is notified that the timeout time has elapsed from the timer unit. This can be installed on the receiver.

As described above, according to the data transmission system of the present invention, even in the case of a stream type data transmission by TCP, when the line state is bad, only the desired extended layer can be reliably reached to the receiving device, thereby enabling the playback of the video of the quality according to the line state. In addition, since the extended layer is determined according to the content of the content video recorded in the video data, it is possible to transmit the best effort type video data and at the same time ensures high quality suitable for the characteristics of the video content in which the video data is recorded. Video playback becomes possible.

BRIEF DESCRIPTION OF THE DRAWINGS It is a typical device block diagram of the data transmission system which concerns on 1st Embodiment of this invention.
2 is a functional block diagram of a data transmission system in the embodiment.
3 is a functional block diagram showing details of a transmission processing unit and a reception processing unit in the embodiment.
4 is a graph showing the operation of the timer unit in the embodiment.
5 is a flowchart showing the operation of the receiving apparatus of the embodiment.
6 is a schematic diagram showing a data structure of scalable coded image data.

EMBODIMENT OF THE INVENTION The 1st Embodiment of this invention is described with reference to drawings.

The reception device 100 of the present embodiment is configured with a transmission device 101 to form a data transmission system for stream-type transmission of video data.

As shown in FIG. 1, the transmitter 101 and the receiver 100 include a CPU 11, a memory 12, an HDD, a flash memory 14, a communication interface 15, and an input / output device 16. As a so-called computer having a), the CPU 11 executes a program stored in the memory 12 to exhibit various functions. The transmitter 101 and the receiver 100 exchange information by a network such as the Internet, and TCP, not RTP, is used as a transmission protocol on the network.

The transmitting apparatus 101 further includes a photographing camera 13 in addition to the above-mentioned enumeration, and is configured to transmit the captured image data to the receiving apparatus 100 after scalable coding. As shown in the functional block diagram of FIG. 2, the transmitting apparatus 101 functions at least as a video encoding unit 3, an encoding control unit 2, a video data storage unit 4, and a transmission processing unit 7. FIG. It is configured to exert.

On the other hand, the receiving apparatus 100 determines whether to retransmit the normal or error in the transmission image packet transmitted from the transmitting apparatus 101, or if there is an error and retransmits the necessary The video is reproduced by decoding (decoding) scalable coded image data. That is, the receiving device 100 functions as at least an image characteristic specifying unit 5, first, an extended layer setting unit 6, a receiving processing unit 8, and an image decoding unit 9 as shown in FIG. It is configured to.

Next, the configuration of each unit described above, scalable encoding of image data, operations during packet transmission and reception, etc. will be described in detail.

First, the transmitter 101 will be described.

The video encoding unit 3 is configured to scalable encode video data picked up by the camera 13, for example, by SVC within the H.264 video codec standard. More specifically, the video encoding unit 3 uses the video data as a base layer composed of the minimum necessary video elements and the encoded video data composed of an enhancement layer capable of improving the image quality by combining the base layer with the base layer. will be.

The encoding control unit 2 divides which layers into which layers to control whether the image encoding unit 3 performs scalable encoding of image data. In the present embodiment, the enhancement layer is further controlled to be divided into three layers, wherein the enhancement layer is a spatial layer contributing to the high definition of the image, a temporal layer contributing to the frame rate improvement of the image, and an S / N ratio improvement of the image. It is composed of S / N ratio layers that contribute to. By combining the base layer and these enhancement layers properly and decoding, video reproduction of a desired quality type is realized.

The video data storage section 4 stores the encoded video data consisting of the base layer and the extended layer encoded by the video encoding section 3, and the layer data representing the layer of the encoded video data.

The transmission processing unit 7 transmits the encoded video data to the receiving device 100 and, in principle, transmits all the transmission video packets for which the reception completion packet transmitted from the receiving device 100 has not been received. It is configured to retransmit.

More specifically, the transmission processor 7 is composed of a layer recognition unit 71, a transmitter 72, and a retransmission processor 73 as shown in FIG. 3.

The layer recognition unit 71 receives encoded image data obtained from the image data storage unit 4 and layer data indicating the layer, and recognizes which layer data is received encoded image data based on the layer data. will be.

The transmitting unit 72 is configured to generate a transmission image packet storing all of the encoded image data and layer data corresponding to the layer identification unit 82 and transmit the packet to the receiving apparatus 100. Based on the layer recognition by the layer recognition unit 71, sequence information on which part of the sequence is a transmission image packet including a base layer, a temporal layer, a spatial layer, and an S / N ratio layer during packet transmission is described. The retransmission processing section 73 is stored.

The retransmission processing unit 73 receives a reception completion packet indicating the transmission video packet received by the reception device 100 from the reception device 100, and the transmission unit 72 transmits the transmission video packet. The transmission unit 72 resends the transmission video packet which does not receive the reception completion packet from the reception apparatus 100 within a predetermined time.

More specifically, the transmission device 101 is configured to perform retransmission processing on all transmission video packets for which the reception completion processing section 8, which will be described later, does not receive the reception completion packet. In other words, the transmitting apparatus 101 has no function of interrupting the retransmission process for the enhancement layer other than the base layer and the preferred enhancement layer.

Next, the receiving apparatus 100 will be described. As shown in FIG. 2, the reception apparatus 100 includes an image characteristic specifying unit 5, an extension layer setting unit 6, a reception processing unit 8, and an image decoding unit 9.

The video characteristic specifying unit 5 specifies the intensity of the content video motion recorded in the video data based on the motion vector information when the video data is scalable encoded by the video encoding unit 3, thereby constructing the video data. It is to specify the image characteristic of. The video characteristic specifying unit 5 also specifies the video characteristic based on the genre of the content video, etc. stored in the meta data of the video data. For example, if the genre is sports or the like, the movement of the content video is determined to be intense. In addition, the video characteristic specifying unit 5 specifies that a large amount of fine descriptions are included in the content video when the high frequency component occurs at a predetermined value or more when the video data is scalable encoded.

The priority enhancement layer setting unit 6 selects any one of a temporal layer, a spatial layer, and an S / N ratio layer based on the intensity of the content image movement, the amount of noise component, the resolution, and the like specified by the image characteristic specifying unit 5. One is to set an extension layer first or which one is to be set to perform retransmission processing or FEC processing to be described later for each extension layer. For example, if the genre of the video content is specified to be violent in sports or the like as described above, the priority enhancement layer setting unit 6 prioritizes the temporal layer so that the frame rate at the time of moving picture reproduction is kept above a predetermined value. Set as an extended layer. In addition, in the case of a high resolution content image in which motion is not violent and many detailed descriptions are included as moving images of other genres, the priority extension layer setting unit 6 sets the spatial layer as an extension layer first. In addition, in the case of a content image having a lot of noise components, the S / N ratio layer is first set as an extension layer.

As shown in Fig. 3, the reception processing unit 8 includes a reception unit 81, a layer identification unit 82, an FEC unit 83, a reception completion processing unit 84, a frame detection unit 85, and a timer unit 86. Transmission of an extended layer set in a base layer and a priority enhancement layer among transmission video packets which have not been received according to a line state or the like, and are damaged and cannot be repaired. It is configured to retransmit only video packets. In addition, the video decoding unit 9 decodes using both layers that can be used based on the transmission video packet received by the reception processing unit 8 and the transmission video packet recovered by the FEC unit 83. The data received by the receiver 81 includes at least a base layer and a first enhancement layer, and combines and decodes them. For example, when the line condition is good, the video decoding unit 9 is configured to decode by combining data of an extension layer other than the enhancement layer first so as to reproduce a higher quality video.

The receiving unit 81 receives each transmission video packet together with the sequence number from the transmitting unit 72.

The layer recognizing unit 71 identifies a layer of coded image data stored in the transmission image packet received by the receiving unit 81 based on the layer data. If it is found that there is damage or omission of the transmitted video packet received by the layer recognition unit 71, the FEC unit 83 attempts FEC processing. Here, the repaired encoded video data is sent to the video decoder 9 to be used for video reproduction.

The reception completion processing unit 84 transmits a reception completion packet indicating the transmission video packet received at the reception unit 81 to the transmission apparatus 101 so that the retransmission processing unit 73 does not start the retransmission processing. The reception completion processing unit 84 also transmits a reception completion packet to the transmission apparatus 101 so that retransmission processing is started for the transmission video packet in which the FEC unit 83 could not error correct among the received transmission video packets. I never do that. In addition, the reception completion processing unit 84 transmits the dummy reception completion packet so that the retransmission process is interrupted for the unreceived transmission video packet in accordance with the operations of the frame detection unit 85 and the timer unit 86 described later. Configured to transmit for. Another detailed operation will be described later.

The frame detection unit 85 includes a base layer reception completion time point that is a time point when the base layer of the reproduction object frame is provided by the transmission video packet received at the reception unit 81 at a time point before the reproduction timing of any reproduction object frame; First of all, the first enhancement layer reception completion point is detected.

More specifically, the frame detection unit 85 temporarily decodes the base layer and the priority enhancement layer by the plurality of transmission video packets received by the reception unit 81, and completes the base layer reception at a time when the temporary decoding succeeds. It is configured to detect the viewpoint and the priority layer reception completion timing.

The timer unit 86 is configured to notify the reception completion processing unit that the timeout time has elapsed when a timeout time elapses from the time point at which the base layer reception completion time and the priority enhancement layer reception completion time elapse. It is.

When the reception completion processing unit 84 is notified that the timeout time has elapsed from the timer unit 86, the reception completion processing unit 84 stops the retransmission processing in the transmission apparatus 101 for the unreceived transmission video packet. and a reception completion packet of a dummy. At this time, the reception completion processing unit 84 is configured to transmit a reception completion packet to the transmission device 101 for an unreceived transmission video packet having a sequence number smaller than the largest number among the sequence numbers of the received transmission video packet. It is.

In this way, when the reproduction timing of the playback target frame cannot be received among the enhancement layers other than the base layer and the preferred enhancement layer, the retransmission process is stopped to maintain the real-time reproduction of the moving picture. In addition, since the reception completion processing unit 84 stops the retransmission processing in accordance with the operation of the frame detection unit 85 and the timer unit 86, the protected base layer and the priority enhancement layer must be received to play the video. It is available. In addition, since the reception completion packet is not transmitted to the other extension layer during the base layer reception completion time or from the lower end of the reception of the priority enhancement layer to the timeout time, the retransmission process can be continued to the transmission apparatus 101. have. Therefore, depending on the line condition, another extension layer may be provided for use in video playback, or the transmission video packet may not be available until the time-out time, and retransmission is interrupted, so that other enhancement layers may not be used for video playback. have. In other words, it is possible to maintain the minimum image quality in user's taste or video characteristics by the protected base layer and the priority extension layer, and to change the use of other extension layers depending on the line conditions. Can be.

Next, an operation when receiving a transmission video packet with respect to a playback target frame in which the reception apparatus 100 configured as described above will be described.

First, as shown in Fig. 4A, a case will be described in which a base frame is first provided from a certain playback target frame to a playback timing, and then an extension layer is first provided. In addition, since the reception processing section 84 is configured to transmit only the reception completed packet of the transmission video packet actually received to the transmission device 101 until the timeout time before the playback timing in FIG. 4 (a) elapses. Retransmission processing is automatically performed from the transmitting apparatus 101 for those that have not been received by the time out.

As shown in the flowchart of Fig. 5, when the reception unit 81 starts receiving the transmission video packet of the frame to be reproduced (step S1), the frame detection unit 85 has already received the layer identification unit ( In step 82, it is attempted whether the temporary encoding succeeds by using all transmission video packets identified as the base layer (step S2).

When the temporary encoding of the base layer is successful by the transmission video packet already received by the receiver 81, the frame detector 85 detects the time point as the base layer reception completion time point.

In addition, the frame detection unit 85 attempts to temporarily decode the enhancement layer by using all transmission video packets that are already received by the reception unit 81 and are identified as the enhancement layer by the layer identification unit 82 ( Step S3).

If the temporary decoding does not succeed here, the frame detection unit 85 adds a newly received transmission video packet by retransmission processing or the like and repeats until the temporary decoding of the enhancement layer succeeds (step S4).

First, the frame detection unit 85 detects the time when the temporary decoding of the enhancement layer is successful as the enhancement layer reception completion time, and the timer 86 starts counting from this time (step S5).

The timer unit 86 first notifies the reception completion processing unit of the time when the timeout time elapses from the completion layer reception completion point of time (step S6), and the reception completion processing unit not received the received video with the notification. The dummy reception completed packet is transmitted to the transmitting apparatus 101 for having a sequence number smaller than the largest number among the received sequence numbers in the packet (step S7).

Next, as shown in Fig. 4 (b), the case where the enhancement layer is provided first and then the base layer is provided until the reproduction timing will be described.

As shown in the left side of the flowchart of FIG. 5, when the temporary encoding of the priority enhancement layer succeeds by the transmission video packet already received by the reception unit 81, the frame detection unit 85 completes the reception of the enhancement layer first. It detects as a viewpoint (step S8). In addition, if the temporal encoding is not successful in both the base layer and the first enhancement layer, the frame detection unit 85 further attempts temporal encoding of each layer by adding a new transmission video packet received by the receiver 81 (step S9).

In addition, the frame detector 85 attempts to temporarily decode the base layer by using all transmission image packets already received by the receiver 81 and identified by the layer identifier 82 as the base layer (step S10). ).

If the temporary decoding is not successful, the frame detection unit 85 repeats until the temporary decoding of the base layer is successful by adding the newly received transmission video packet by retransmission processing or the like (step S11).

When the temporary decoding of the base layer succeeds, the frame detection unit 85 detects the base layer reception completion time, and from this point on, the timer unit 86 starts counting (step S5).

When the timeout time has elapsed from the base layer reception completion time, the timer unit 86 notifies the reception completion processing unit of this (step S6), and the reception completion processing unit of the unreceived transmission video packet with the notification. A dummy reception completion packet is transmitted to the transmitting apparatus 101 with respect to one having a sequence number smaller than the largest one among the received sequence numbers (step S7).

As the receiver 100 operates as described above, a dummy reception completion packet is transmitted to an unreceived extended layer in a state where at least a base layer and a preferred extension layer are provided regardless of the order of reception completion. In other words, even if the retransmission process by the transmitting apparatus 101 is forcibly interrupted by the dummy received packet, the base layer and the priority enhancement layer are always secured so as to ensure the desired video quality while not impairing real-time. .

In addition, retransmission processing continues until the timeout time for the other enhancement layers, so that playback timing may be provided depending on the line state, and new image quality improvement may be expected depending on the line condition.

In other words, the receiver 100 according to the present embodiment is capable of high quality video playback provided with all the enhancement layers when the line condition is good, while the quality is degraded because some extension layers are missing when the line condition is bad. However, it is possible to play the video with real time while securing the base layer and the extended layer. In other words, even a combination of a transmission protocol presumed to be retransmission processing such as TCP and scalable coding, it is possible to reproduce video of image quality according to a line condition and provide a best effort type service.

In addition, the reception apparatus 100 is a priority layer which is an extended layer to be retransmitted based on the image characteristic specifying unit 5 specifying the image characteristic of the image data and the image characteristic specified by the image characteristic specifying unit 5. The expansion layer setting unit 6 for setting the extension layer is provided so that the extension layer corresponding to the video characteristics of the image data can be retransmitted, and the extension layer and the base layer always present in the content image are combined regardless of the line condition. You can play a movie.

That is, since the priority layer is set in accordance with the characteristics of the priority content image rather than the line state, transmission of an appropriate extension layer is always protected for each image data, so that the quality of video playback can be improved than before.

Next, a second embodiment of the present invention will be described.

In the first embodiment, the reception completion processing unit 84 is configured to stop the retransmission process when the timeout time elapses from the time when both the base layer and the priority enhancement layer are provided, but in the second embodiment, Regardless of whether the enhancement layer is provided, a dummy reception completion packet for the unreceived transmission video packet is transmitted after the timeout time elapses from the completion of the base layer reception, which is the completion of the base layer.

More specifically, the frame detection unit 85 is configured to enforce whether the temporary encoding succeeds only for the base frame, and the timer unit 86 always starts counting from the base layer reception completion time point, so that the reception completion processing unit ( 84), the timeout time elapsed.

According to this second embodiment, the base frame can be protected, and real time can be secured with minimum image quality. In addition, since the transmitting apparatus 101 continues the retransmission process of the enhancement layer until the timeout time elapses, the possibility that an enhancement layer is provided may be increased depending on the line condition. That is, the result of the best effort type in which the line condition and the image quality are linked can also be obtained by the receiving apparatus 100 of the second embodiment.

Other embodiments will be described.

In the above embodiment, the priority extension layer setting unit sets only one extension layer as the priority extension layer, but for example, a plurality of extension layers may be set as the priority extension layer. In addition, the priority extension layer setting unit may be configured to give priority to the priority of each extension layer according to the specified video characteristic.

In the above embodiment, although the video data captured by the camera is scalable encoded in the video encoding unit in order, it may be transmitted to the receiving apparatus that the encoded data encoded in the scalable encoding is stored in the video data storage unit.

In the case where the original receiving terminal is a terminal having a display resolution capability of VGA (640x480) level, such as a receiving capability of the receiving terminal, for example, a resolution, a space in which the transmission image source resolution is full HD (1920 x 1080) resolution is used. Sending enhancement layer data does not fully utilize the function. In addition, although the decoding capability of the decoder of the receiver supports only 720P (1280 × 720), even if transmitting the spatial expansion layer data in full HD (1920 × 1080) resolution cannot fully utilize the function, By negotiation, the capacities such as the maximum resolution, the bit rate, etc. of the receiving terminal are exchanged, and the transmitting terminal may preempt and transmit the enhancement layer data in advance so that a video below the capability is obtained. The techniques described so far can be fully utilized even with this limitation.

In addition, the above-described parts may be installed in the existing transmitter and receiver as new programs to realize the functions of the present invention. Scalable coding in the video encoding unit is not limited to that shown in the above embodiment. For example, the layer of the enhancement layer may be further divided into a plurality of layers.

Other modifications and combinations of the embodiments may be made without departing from the spirit of the invention.

200: data transmission system
100: receiver
101: transmission device
2: coding control unit
3: video encoder
4: image data storage
5: Image Characterization Part
6: extension layer setting first
7: Transmission processing unit
8: reception processing unit
9: video decoder
71: layer recognition unit
72: sending part
73: retransmission processing unit
81: receiver
82: layer identification unit
83: FEC section
84: reception completion processing unit
85: frame detector
86: timer unit

Claims (15)

In a receiving apparatus for receiving a transmitted video packet,
A frame detector which detects a base layer reception completion time, which is a time when a base layer of a reproduction target frame is prepared by the received transmission video packet;
A timer unit for notifying that the timeout time has elapsed when the predetermined timeout time elapses from the time when the base layer of the frame to be reproduced is provided by the frame detector;
And a reception completion processing unit for transmitting a reception completion packet to a transmission device for an unreceived transmission video packet when it is notified that the timeout time has elapsed from the timer unit. The apparatus of claim 1, further comprising: a receiver configured to receive the transmission video packet and the sequence number from the transmission device;
A layer identification unit identifying a layer of the transmission image packet received by the reception unit based on layer data;
And a FEC unit for performing FEC processing when it is recognized that there is damage or omission of a transmission video packet received by the layer recognition unit. The receiving device according to claim 1, wherein the timeout time is configured to end before a reproduction timing of a reproduction target frame. The method according to claim 1, further comprising a priority extension layer setting unit for setting a priority extension layer which is an extension layer to be retransmitted,
The frame detection unit is configured to detect a priority extension layer reception completion time that is a time when the base layer reception completion time and a priority enhancement layer are provided,
And the timer unit is configured to notify the reception completion processing unit that the timeout time has elapsed when a timeout time elapses from the slower time point among the base layer reception completion time and the priority enhancement layer reception completion time. . The base layer reception completion time or the priority enhancement layer of claim 1, wherein the frame detection unit temporarily decodes the base layer or the priority enhancement layer by using the transmission image packet received by the reception unit, and determines that the temporary decoding succeeds. Receiving device at the time of arrival. The dummy reception completed packet according to claim 1, wherein when the reception completion processing unit is notified that the timeout time has elapsed from the timer unit, the reception completion packet of the dummy is interrupted so that retransmission processing at the transmitting apparatus is stopped for the unreceived transmission video packet. Receiver configured to transmit. The received completion processing unit of claim 1, wherein the reception completion processing unit has a sequence number smaller than the largest number among the sequence numbers of the received transmission video packets at the slower time of the base layer reception completion time or the first enhancement layer reception completion time. And a reception device configured to transmit a reception completion packet to the transmission device for an unreceived transmission video packet. The apparatus of claim 4, further comprising an image characteristic specifying unit which specifies an image characteristic of the image data,
And the priority extension layer setting unit is configured to set a priority extension layer which is an extension layer to be retransmitted based on the image characteristic specified by the image characteristic specifying unit. A data transmission system for transmitting video data in stream type data, comprising: a reception device for receiving a transmission video packet transmitted from a transmission device,
An image data storage unit for storing the encoded image data including the base layer and the enhancement layer and the layer data representing the layer of the encoded image data, and a transmission image storing the layer data corresponding to the encoded image data. A transmitter for generating a packet and transmitting a packet to the receiving device, and receiving a received completion packet indicating a transmission video packet received at the receiving device from the receiving device and not receiving the received completion packet from the receiving device; It is provided with a retransmission process part which retransmits a transmission video packet to the said transmission part,
The receiving device,
A reception completion processing unit for transmitting a reception completion packet indicating at least the transmission video packet received at the reception unit to the transmission apparatus;
A layer identifier for identifying a layer of the transmission image packet received by the receiver based on layer data;
A frame detector which detects a base layer reception completion time, which is a time when a base layer of a reproduction target frame is provided by the transmission image packet received by the reception unit;
A timer unit for notifying the reception completion processing unit that the timeout time has elapsed when a predetermined timeout time elapses from the base layer reception completion time point;
And the reception completion processing section is configured to send a reception completion packet to the transmission apparatus for an unreceived transmission video packet when the timer section has notified that the timeout time has elapsed. The receiving device according to claim 9, wherein the timeout time is configured to end before a reproduction timing of a reproduction target frame. 10. The method of claim 9, further comprising a priority extension layer setting unit for setting a priority extension layer that is an extension layer to be retransmitted,
The frame detection unit is configured to detect a priority extension layer reception completion time that is a time when the base layer reception completion time and a priority enhancement layer are provided,
And the timer unit is configured to notify the reception completion processing unit that the timeout time has elapsed when a timeout time elapses from the slower time point among the base layer reception completion time and the priority enhancement layer reception completion time. . 10. The method of claim 9, wherein the frame detector is configured to temporarily decode the base layer or the priority enhancement layer by using the transmission video packet received by the receiver, and to determine the time at which the temporary decoding succeeds. Receiving device at the time of arrival. 10. The method of claim 9, wherein the reception completion processing unit has a sequence number smaller than the largest number among the sequence numbers of the received transmission video packet at the slower of the base layer reception completion time or the reception of the priority enhancement layer reception time. And a reception device configured to transmit a reception completion packet to the transmission device for an unreceived transmission video packet. The apparatus of claim 11, further comprising an image characteristic specifying unit which specifies an image characteristic of the image data,
And the priority extension layer setting unit is configured to set a priority extension layer which is an extension layer to be retransmitted based on the image characteristic specified by the image characteristic specifying unit. In a data transmission system for transmitting video data in stream type data, there is provided a program for a reception device used for a reception device for receiving a transmission video packet transmitted from a transmission device.
The transmission device transmits a video data storage unit for storing encoded video data including a base layer and an extended layer and layer data representing a layer of the encoded video data, and layer data corresponding to the encoded video data. A transmitter for generating a video packet and transmitting the packet to the receiving device, and receiving a reception completion packet indicating a transmission video packet received at the receiving device from the receiving device, and not receiving the received completion packet from the receiving device. It is provided with a retransmission process part which retransmits a transmission video packet to the said transmission part,
The receiving device program,
A reception completion processing unit for transmitting a reception completion packet indicating at least the transmission video packet received at the reception unit to the transmission apparatus;
A layer identifier for identifying a layer of the transmission image packet received by the receiver based on layer data;
A frame detector which detects a base layer reception completion time, which is a time when a base layer of a reproduction target frame is provided by the transmission image packet received by the reception unit;
And a timer unit for notifying the reception completion processing unit that the timeout time has elapsed when a predetermined timeout time elapses from the time when the base layer of the playback target frame is provided by the frame detector,
And the reception completion processing section is configured to send a reception completion packet to the transmission apparatus for an unreceived transmission video packet when it is notified that the timeout time has elapsed from the timer section.

Images