JP5947631B2 - Receiving device and program for receiving device - Google Patents

Description

  The present invention relates to a receiving apparatus and a receiving apparatus program for receiving a transmission video packet transmitted from a transmitting apparatus in a data transfer system for transferring video data in a stream type.

  For example, when video data is transferred via Internet communication, it is not a download type transmission method, but stream type data that is also played back in parallel while data is being transferred from the transmitting device to the receiving device. Transfer is used.

  When such stream-type data transfer of video data is performed, encoded video data obtained by scalable-coding video data is used so that a transmission video result corresponding to a line state, receiving terminal capability, and the like is obtained.

  An example of scalable coding is H.264. SVC (Scalable Video Coding) within the H.264 video codec standard. As shown in FIG. 6, the data structure of this codec can be divided into a base layer composed of minimum necessary moving picture elements and an extended layer that can improve video quality by combining with the base layer. .

  The base layer, which is a highly reliable channel, is a data stream indispensable for reproducing video on the receiving side, and is therefore transmitted and received with high reliability by redundancy such as FEC (Forward Error Correction).

  On the other hand, the enhancement layer, which is a low-reliability channel, includes a spatial layer that contributes to higher definition of images, a time layer that contributes to improvement of the frame rate of moving images, and an S / N that contributes to improvement of the S / N ratio of images. It is further divided into N ratio layers.

  For example, since the FEC processing is not applied to the enhancement layer, it becomes difficult to reach the receiving device when the line condition deteriorates. In the receiving device, the essential base layer is always received, so the transmission video that matches the channel quality can be obtained by combining the base layer with the one that has been safely received from the transmitting device and decoding appropriately. Results are obtained.

  By the way, when performing data transfer using encoded video data that has been encoded by scalable coding, RTP (Real Time Protocol), which is a transmission method in which retransmission processing is not performed, is used as a transfer protocol as disclosed in Patent Document 1 and the like. Therefore, the real-time property is prioritized over the reliability of data transfer.

  Patent Document 1 discloses a data transfer system configured to perform retransmission processing for some layers while using RTP that is not subjected to retransmission processing as a transfer protocol in principle.

  However, when performing stream-type transfer of moving images or the like, TCP that maintains reliability by performing retransmission processing may be used as a transfer protocol. When encoded video data that is scalable-coded by TCP is transferred, if there is a transmission video packet that has not been received by the receiving apparatus, retransmission processing is performed for all unreachable transmission video packets. For this reason, when scalable encoded data is transferred by TCP, the transmission video packet of the enhancement layer that may not be reached in the receiving apparatus is ensured if the purpose is only to reproduce the moving picture in the worst case. Will be retransmitted. Therefore, although the quality at the time of moving image reproduction can be fixed to the highest quality, the waiting time until all transmission video packets are received occurs, so the real-time property is impaired.

  In other words, simply combining TCP and scalable coding cannot automatically obtain image quality commensurate with the line quality, and it is possible to obtain a best-effort type result as when RTP and scalable coding are combined. Absent. Therefore, the SVC characteristics cannot be exhibited, and it is necessary to perform encoding at a bit rate corresponding to the line state as in the past, or to prepare a plurality of encoded video data.

  Further, the data transfer system disclosed in Patent Document 2 is based on the assumption that the transmission device is configured to repeat retransmission processing unless it receives a reception completion packet from the reception device for the transmitted transmission video packet. In the configuration, the base layer is configured to be surely received by the receiving apparatus, and other enhancement layers that cannot be received are not uniformly retransmitted.

  More specifically, the receiving apparatus does not transmit a reception completion packet to the transmitting apparatus as usual for those that have not been received for the base layer, and is actually used for those that cannot be received among the enhancement layers. Is configured to forcibly terminate the retransmission process for the enhancement layer by transmitting a reception completion packet to the transmission device even in a state of not receiving.

  However, in such a receiving apparatus, when there is even one enhancement layer transmission video bucket that could not be received by the receiving apparatus, the retransmission process of the enhancement layer is aborted, and thus the line condition is deteriorated. The enhancement layer can hardly be decoded, and there may be a case where only the base layer is transmitted.

  In other words, such a receiving apparatus protects the priority enhancement layer even if there is a request to perform video playback while protecting at least the base layer and another priority enhancement layer, regardless of the line state. It is impossible to provide a best-effort service while reflecting the user's preference such as image quality.

Japanese Patent No. 3757857 JP-A-9-307510

  Therefore, the present invention has been made in view of the above-described problems, and the characteristics of scalable coding are exhibited even in the case of stream type data transfer by TCP, and transfer of best effort type video data is enabled. Another object of the present invention is to provide a receiving apparatus and a receiving apparatus program capable of protecting a part of an enhancement layer other than the base layer while ensuring real-time properties. In other words, according to the present invention, although a transmission device such as a server has a retransmission function, even if it does not have a configuration for setting which layer to retransmit, only the configuration on the receiving device side that is a client Therefore, it is an object of the present invention to be able to receive a best-effort service while protecting a part of the base layer and the enhancement layer.

  That is, the present invention provides a receiving apparatus for receiving a transmission video packet transmitted from a transmitting apparatus in a data transfer system for transferring video data in stream type data, wherein the transmitting apparatus includes a code composed of a base layer and an enhancement layer. A video data storage unit that stores a pair of layer data indicating a layer of the encoded video data and the encoded video data; and a transmission video packet that stores both the encoded video data and the corresponding layer data. A transmission unit that transmits a packet to the receiver and a reception completion packet indicating a transmission video packet received by the reception device from the reception device, and a transmission video packet that has not received the reception completion packet from the reception device It is assumed that the transmission unit includes a retransmission processing unit that retransmits the message.

  The reception device transmits at least a reception completion packet indicating a transmission video packet received by the reception unit to the transmission device, and a layer of the transmission video packet received by the reception unit is layered. A layer identifying unit for identifying based on data, a frame detecting unit for detecting a base layer reception completion point when a base layer of a frame to be reproduced is aligned by transmission video packets received by the receiving unit, and the base layer A timer unit that notifies the reception completion processing unit that a timeout time has elapsed when a predetermined timeout time has elapsed since the reception completion time point, and the reception completion processing unit receives a timeout time from the timer unit. When it is notified that the elapse of the Characterized in that it is configured to transmit the received completion packet to the transmitting device are.

  In such a case, when the reception completion processing unit is notified from the timer unit that a timeout time has elapsed, the reception completion processing unit transmits a reception completion packet to the transmission device for an unreceived transmission video packet. Even if the data transfer is performed by a transfer protocol having a retransmission process such as TCP, even if the line condition is bad, only the base layer is reliably received by the retransmission process. Can abort the retransmission process.

  Moreover, the enhancement layer retransmission processing is aborted after the timeout time elapses from the completion of the base layer reception, so that the unreceived enhancement layer is also retransmitted at least until the timeout time elapses. It can be protected by processing.

  That is, the receiving apparatus of the present invention does not transmit the reception completion packet uniformly for the enhancement layer that has not been received and aborts the retransmission processing by the transmitting apparatus, but has a grace period called a timeout period, It is possible to protect a part of these.

  On the other hand, when the line condition is good, all the enhancement layers are received by the receiving apparatus, and reproduction with the highest quality is possible.

  Therefore, according to the present invention, even if there is no special configuration in the transmission apparatus, even with a transfer apparatus having retransmission processing such as TCP, even with no special configuration in the transmission apparatus, it is possible to reproduce a moving image with an image quality corresponding to the line condition. It is possible to provide a best-effort service. In addition, it is possible to provide a best-effort service that reflects a user's request for image quality, such as protecting not only the base layer but also a part of the enhancement layer.

  In order to protect a part of the enhancement layer in the same manner as the base layer and not to impair the real-time performance of the video playback, the time-out period should be terminated before the playback timing of the playback target frame. It suffices to be configured.

  In order to protect a part of the enhancement layer as a priority enhancement layer and ensure that the priority enhancement layer is received, a priority enhancement layer setting unit that sets a priority enhancement layer that is an enhancement layer to be retransmitted is further provided. The frame detection unit is configured to detect the base layer reception completion time point and a priority enhancement layer reception completion time point when the priority enhancement layer is aligned, and the timer unit is configured to detect the base layer reception. What is necessary is just to be configured to notify the reception completion processing unit that the timeout time has elapsed when the timeout time elapses from the later time point between the completion time point and the priority enhancement layer reception completion time point. If this is the case, the base layer and the priority enhancement layer are protected by the retransmission process and received by the receiving apparatus, and the retransmission process is terminated for the extension layer that has not been received when the timeout time elapses. It is possible to play a video that matches the situation.

  In order to be able to detect whether the base layer and the priority enhancement layer have been received by the configuration of the receiving device without having to transmit special data from the transmitting device, the frame detection unit includes A base layer or a priority enhancement layer is provisionally decoded using a transmission video packet received by a reception unit, and a time when the provisional decoding is successful is configured as the base layer reception completion time or the priority enhancement layer arrival time. It only has to be.

  Even if a detailed list of transmission video packets or the like is not transmitted from the transmission device to the reception device, the retransmission process can be terminated for data other than the base layer and the priority enhancement layer with a simple configuration of only the reception device. For example, the reception completion processing unit may determine whether the base layer reception completion time point or the priority enhancement layer reception completion time point is the later one of the sequence numbers of the transmission video packets being received. What is necessary is just to be comprised so that a reception completion packet may be transmitted to the said transmission apparatus about the unreceived transmission video packet which has a small sequence number.

  In order to add a function as a receiving device of the present invention to an existing receiving device so that a best-effort result can be obtained at the time of moving image reproduction, at least a transmission video packet received by the receiving unit is used. A reception completion processing unit that transmits a reception completion packet to the transmission device, a layer identification unit that identifies a layer of a transmission video packet received by the reception unit based on layer data, and a transmission received by the reception unit A frame detection unit that detects a base layer reception completion point when the base layer of the playback target frame is aligned by the video packet, and a time-out time that is predetermined from the time when the base layer of the playback target frame is aligned in the frame detection unit If the timeout period has passed to the reception completion processing unit, And a reception completion processing unit configured to transmit a reception completion packet to an unreceived transmission video packet to the transmission device when notified from the timer unit that a timeout period has elapsed. What is necessary is just to install the program for receivers characterized by being installed in the existing receiver.

  As described above, according to the data transfer system and the like of the present invention, even in the case of the stream type data transfer by TCP, when the line state is bad, only the desired enhancement layer is surely reached the receiving apparatus, and the line state is set. It is possible to play a video with the corresponding quality. In addition, since the priority extension layer is determined according to the contents of the content video recorded in the video data, it is possible to transfer the best-effort video data and record the video data while ensuring real-time performance. High-quality video playback that matches the characteristics of the content video being played is possible.

1 is a schematic device configuration diagram of a data transfer system according to a first embodiment of the present invention. FIG. The functional block diagram of the data transfer system in the embodiment. The functional block diagram shown about the detail of the transmission process part in the same embodiment, and a reception process part. The graph shown about operation | movement of the timer part in the embodiment. 7 is a flowchart showing the operation of the receiving apparatus of the embodiment. The schematic diagram which shows the data structure of the video data by which the scalable encoding was carried out.

  A first embodiment of the present invention will be described with reference to the drawings.

  The receiving apparatus 100 of the present embodiment forms a data transfer system for paired with the transmitting apparatus 101 to transfer video data in a stream type.

  The transmission device 101 and the reception device 100 are so-called computers having a CPU 11, a memory 12, an HDD or flash memory 14, a communication interface 15, an input / output device 16, etc., as shown in FIG. The CPU 11 executes various programs to exhibit various functions. The transmitting apparatus 101 and the receiving apparatus 100 exchange information via a network such as the Internet, and TCP is used as a transfer protocol on the network instead of RTP.

  The transmission apparatus 101 further includes a shooting camera 13 other than the above-described ones, and is configured to transmit the captured video data to the reception apparatus 100 after performing scalable coding. The transmitting apparatus 101 exhibits at least functions as a video encoding unit 3, an encoding control unit 2, a video data storage unit 4, and a transmission processing unit 7, as shown in the functional block diagram of FIG. It is configured.

  On the other hand, the receiving apparatus 100 determines whether or not to retransmit a transmission video packet transmitted from the transmitting apparatus 101 that has not normally arrived at the receiving apparatus 100 or has an error. Is to replay a moving image by decoding (decoding) the video data that has been scalable-encoded after being retransmitted. That is, as shown in FIG. 2, the receiving apparatus 100 is configured to exhibit at least functions as a video characteristic specifying unit 5, a priority enhancement layer setting unit 6, a reception processing unit 8, and a video decoding unit 9.

  Next, the configuration of each unit described above, scalable encoding of video data, operation in packet transmission / reception, and the like will be described in detail.

  First, the transmission apparatus 101 will be described.

  The video encoding unit 3 is, for example, H.264. The video data captured by the camera 13 is scalable encoded by SVC within the H.264 video codec standard. More specifically, the video encoding unit 3 includes a base layer composed of minimum required moving image elements of video data, and an extension layer that can improve video quality by combining with the base layer. The encoded video data is obtained as follows.

  The encoding control unit 2 controls which layer is divided into how many layers and the video encoding unit 3 performs scalable encoding of video data. In the present embodiment, the enhancement layer is controlled to be further divided into three layers, and the enhancement layer is a spatial layer that contributes to high definition of the image, a temporal layer that contributes to an improvement in the frame rate of the image, and an S of the image. The S / N ratio layer contributes to the improvement of the / N ratio. By decoding after appropriately combining the base layer and these enhancement layers, moving image reproduction with a desired quality aspect is realized.

  The video data storage unit 4 stores encoded video data composed of a base layer and an enhancement layer encoded by the video encoding unit 3 and layer data indicating a layer of the encoded video data in pairs. .

  The transmission processing unit 7 transmits encoded video data to the receiving device 100 as a packet, and in principle, transmits all transmission video packets for which reception completion packets transmitted from the receiving device 100 have not been received. It is configured to retransmit.

  More specifically, the transmission processing unit 7 includes a layer recognition unit 71, a transmission unit 72, and a retransmission processing unit 73 as shown in FIG.

  The layer recognizing unit 71 receives the encoded video data acquired from the video data storage unit 4 and layer data indicating the layer, and determines which layer the received encoded video data is based on the layer data. To recognize.

  The transmission unit 72 is configured to generate a transmission video packet stored together with the encoded video data recognized by the layer identification unit 82 and the corresponding layer data, and transmit the packet to the reception device 100. . Then, based on the layer recognition in the layer recognizing unit 71, sequence information about which part of the sequence is a transmission video packet including a base layer, a temporal layer, a spatial layer, and an S / N ratio layer at the time of packet transmission. The data is stored in the retransmission processing unit 73.

  The retransmission processing unit 73 receives a reception completion packet indicating a transmission video packet received by the receiving device 100 from the receiving device 100, and a predetermined time after the transmission unit 72 transmits the transmission video packet. The transmission unit 72 is configured to retransmit the transmission video packet that has not received the reception completion packet from the receiving device 100.

  More specifically, the transmission apparatus 101 is configured to perform retransmission processing for all transmission video packets that have not received a reception completion packet from a reception completion processing unit 8 described later. That is, the transmitting apparatus 101 does not have a function of canceling retransmission processing for an enhancement layer other than the base layer and the priority enhancement layer.

  Next, the receiving apparatus 100 will be described. As shown in FIG. 2, the receiving device 100 includes a video characteristic specifying unit 5, a priority enhancement layer setting unit 6, a reception processing unit 8, and a video decoding unit 9.

  The video characteristic specifying unit 5 is configured to specify the intensity of motion of the content video recorded in the video data based on motion vector information when the video data is scalable encoded in the video encoding unit 3 This specifies the video characteristics of the video data. The video characteristic specifying unit 5 specifies the video characteristics based on the genre of the content video stored in the metadata of the video data. For example, if the genre is sports or the like, it is specified that the motion of the content video is intense. Further, the video characteristic specifying unit 5 specifies that the content video includes many detailed descriptions when a high-frequency component is generated at a predetermined value or more when the video data is scalable encoded. To do.

  The priority extension layer setting unit 6 includes a temporal layer, a spatial layer, and an S / N ratio layer based on the intensity of motion of the content video specified by the video characteristic specifying unit 5, the amount of noise components, the resolution, and the like. It is set which is used as a priority enhancement layer, or what weight is used to perform retransmission processing or FEC processing described later on each enhancement layer. For example, as described above, when the genre of the video content is specified to be a thing with intense movement such as sports, the priority extension layer setting unit 6 keeps the frame rate at the time of moving image reproduction at a predetermined value or more. The time layer is set as the priority extension layer so that it can bend. In addition, in the case of a high-resolution content video that is a moving image of another genre and does not move so much and includes many detailed depictions, the priority extension layer setting unit 6 sets the spatial layer as a priority extension layer. Set as. Furthermore, in the case of a content video with a lot of noise components, the S / N ratio layer is set as a priority extension layer.

  As shown in FIG. 3, the reception processing unit 8 includes a reception unit 81, a layer identification unit 82, an FEC unit 83, a reception completion processing unit 84, a frame detection unit 85, and a timer unit 86. The transmission video packet transmitted from 101 is received, and the transmission video packet of the extension layer set in the base layer and the priority enhancement layer among the transmission video packet that has not arrived due to the line state or the like and the transmission video packet that is broken and cannot be repaired It is configured to retransmit only. Further, the video decoding unit 9 performs decoding using all layers that can be used based on the transmission video packet received by the reception processing unit 8 and the transmission video packet restored by the FEC unit 83. The data received by the receiving unit 81 includes at least a base layer and a priority enhancement layer, and decoding is performed by combining these. For example, when the line condition is good, the video decoding unit 9 is configured to perform decoding by combining data of enhancement layers other than the priority enhancement layer, and to further reproduce high-quality moving images.

  The receiving unit 81 receives each transmission video packet together with a sequence number from the sending unit 72.

  The layer recognizing unit 71 identifies a layer of encoded video data stored in a transmission video packet received by the receiving unit 81 based on the layer data. When it is found that the transmission video packet received by the layer recognition unit 71 is damaged or missing, the FEC unit 83 tries the FEC process. Here, the restored encoded video data is sent to the video decoding unit 9 and used for moving image reproduction.

  The reception completion processing unit 84 transmits a reception completion packet indicating the transmission video packet received by the reception unit 81 to the transmission apparatus 101 so that the retransmission processing unit 73 does not start the retransmission process. In addition, the reception completion processing unit 84 sends a reception completion packet to the transmission apparatus 101 so that retransmission processing is started for transmission video packets that could not be error corrected in the FEC unit 83 among the received transmission video packets. Do not send. Further, the reception completion processing unit 84 sends a dummy reception completion packet to the transmission apparatus 101 so that the retransmission processing is terminated for the unreceived transmission video packet in accordance with the operation of the frame detection unit 85 and the timer unit 86 described later. To transmit. Further detailed operations will be described later.

  The frame detection unit 85 includes a base layer reception completion time point that is a time point when a base layer of a reproduction target frame is aligned by a transmission video packet received by the reception unit 81 at a time point before a reproduction timing of a certain reproduction target frame. The priority enhancement layer reception completion time point, which is the time point when the priority enhancement layers are arranged, is detected.

  More specifically, the frame detection unit 85 performs provisional decoding of the base layer and the priority enhancement layer using a plurality of transmission video packets received by the reception unit 81, and receives base layer reception at the time when the provisional decoding is successful. A completion time point and a priority enhancement layer reception completion time point are detected.

  The timer unit 86 notifies the reception completion processing unit that the timeout period has elapsed when a timeout period elapses from the later of the base layer reception completion point and the priority enhancement layer reception completion point. It is configured.

  Then, when the reception completion processing unit 84 is notified from the timer unit 86 that the timeout time has elapsed, the dummy reception completion is performed so that the retransmission processing in the transmission apparatus 101 is terminated for the unreceived transmission video packet. It is configured to transmit a packet. At this time, the reception completion processing unit 84 transmits a reception completion packet to the transmission apparatus 101 for an unreceived transmission video packet having a sequence number smaller than the largest number among the sequence numbers of the transmission video packets being received. It is comprised so that it may do.

  In this way, for the extension layers other than the base layer and the priority enhancement layer, those that could not be received before the playback timing of the playback target frame, the real-time performance of the video playback is maintained by terminating the retransmission process. is there. In addition, since the reception completion processing unit 84 cancels the retransmission process in accordance with the operations of the frame detection unit 85 and the timer unit 86, the reception of the protected base layer and the priority enhancement layer is always received. And can be used for moving image reproduction. Further, since the reception completion packet is not transmitted for the other enhancement layers from the later of the base layer reception completion time or the priority enhancement layer reception completion time to the timeout time, the retransmission processing is continued to the transmission apparatus 101. Can be made. Therefore, depending on the line status, other enhancement layers may be used together for video playback, and other video transmissions will not be completed before the timeout time, and the retransmission process will be aborted, so other enhancement layers will not be used for video playback. In some cases. In other words, the protected base layer and the priority enhancement layer can maintain the minimum video quality according to the user's preference and video characteristics, and whether other extension layers are used according to the line conditions. Can be changed to a best-effort service.

  Next, the operation when receiving a transmission video packet for a certain playback target frame of the receiving apparatus 100 configured as described above will be described.

  First, as shown in FIG. 4 (a), a case will be described in which base frames are aligned first by the playback timing in a certain playback target frame, and then priority enhancement layers are aligned thereafter. In FIG. 4A, the reception processing unit 84 transmits only the reception completion packet of the actually received transmission video packet to the transmission apparatus 101 until a certain timeout time elapses before the reproduction timing. Therefore, a retransmission process is automatically performed from the transmission apparatus 101 for those that have not been received until the timeout time elapses.

  As shown in the flowchart of FIG. 5, when reception of the transmission video packet of the reproduction target frame is started in the reception unit 81 (step S1), the frame detection unit 85 has already received the layer identification unit. It is tried whether or not the temporary encoding succeeds using all the transmission video packets identified as the base layer in 82 (step S2).

  When the base layer temporary encoding is successful with the transmission video packet already received by the reception unit 81, the frame detection unit 85 detects that time as the base layer reception completion time.

  Further, the frame detecting unit 85 performs provisional decoding of the priority enhancement layer using all transmission video packets that have been received by the receiving unit 81 and identified by the layer identifying unit 82 as a priority enhancement layer. Try (step S3).

  Here, if the temporary decoding is not successful, the frame detection unit 85 adds a transmission video packet newly received by retransmission processing or the like and repeats until the temporary extension layer temporary decoding is successful (step S4). .

  The frame detection unit 85 detects the time when the priority enhancement layer temporary decoding is successful as the time when the priority enhancement layer reception is completed, and the timer unit 86 starts counting from this time (step S5).

  The timer unit 86 notifies the reception completion processing unit when a time-out period has elapsed since the completion of the priority enhancement layer reception (step S6), and the reception completion processing unit transmits an unreceived transmission with the notification. A dummy reception completion packet is transmitted to the transmission apparatus 101 for a video packet having a sequence number smaller than the largest received sequence number (step S7).

  Next, as shown in FIG. 4B, a case will be described in which the priority extension layers are aligned before the reproduction timing and the base layers are aligned thereafter.

  As shown in the left half of the flowchart of FIG. 5, when the priority enhancement layer has been successfully encoded by the transmission video packet already received by the reception unit 81, the frame detection unit 85 receives the priority enhancement layer reception at that time. It detects as completion time (step S8). If neither the base layer nor the priority enhancement layer succeeds in temporary encoding, the frame detecting unit 85 tries to temporarily encode each layer by adding a transmission video packet newly received by the receiving unit 81. (Step S9).

  Further, the frame detection unit 85 attempts to temporarily decode the base layer using all the transmission video packets that have already been received by the reception unit 81 and have been identified as the base layer by the layer identification unit 82 ( Step S10).

  If the temporary decoding is not successful, the frame detection unit 85 adds a transmission video packet newly received by retransmission processing or the like and repeats until the temporary decoding of the base layer is successful (step S11).

  The frame detection unit 85 detects the time when the base layer temporary decoding is successful as the base layer reception completion time, and the timer unit 86 starts counting from this time (step S5).

  The timer unit 86 notifies the reception completion processing unit when a timeout time has elapsed from the completion of base layer reception (step S6), and the reception completion processing unit notifies the transmission video that has not been received with the notification. A dummy reception completion packet is transmitted to the transmission apparatus 101 for packets having a sequence number smaller than the largest received sequence number (step S7).

  By operating the receiving apparatus 100 in this way, a dummy reception completion packet is transmitted for an unreceived extension layer in a state where at least the base layer and the priority extension layer are aligned, regardless of the order of reception completion. That is, even if retransmission processing by the transmission apparatus 101 is forcibly terminated by a dummy reception completion packet, the base layer and the priority enhancement layer are always secured, and real-time performance is ensured while ensuring desired video quality. There is no loss.

  In addition, since the retransmission process is continued until the timeout time for other enhancement layers, depending on the line state, it may be arranged by the reproduction timing, and further improvement in video quality can be expected according to the line state.

  In other words, the receiving apparatus 100 according to the present embodiment can perform high-quality video reproduction with all the enhancement layers when the line state is good as described above, while some extension layers when the line state is bad. Although it is lost and the quality is deteriorated, it is possible to reproduce the moving image with the real-time property secured while securing the base layer and the priority extension layer. In other words, even with a combination of a transfer protocol based on retransmission processing such as TCP and scalable coding, it is possible to reproduce a moving image with an image quality according to the line status and provide a best effort type service.

  Further, the receiving apparatus 100 includes a video characteristic specifying unit 5 that specifies video characteristics of video data, and a priority extension layer that is an extension layer to be retransmitted based on the video characteristics specified by the video characteristic specifying unit 5. Since the priority extension layer setting unit 6 for setting is provided, the extension layer according to the video characteristics of the video data can be retransmitted, and the extension layer and the base layer always matching the content video regardless of the line status You can play a video that combines.

  In other words, the priority layer is set according to the characteristics of the content video, not the line status. Therefore, transmission of the extension layer suitable for each video data can always be protected, and the quality during video playback has been improved. Than can be raised.

  Next, a second embodiment of the present invention will be described.

  In the first embodiment, the reception completion processing unit 84 is configured to abort the retransmission process when a timeout time elapses from the time when both the base layer and the priority enhancement layer are aligned. In the second embodiment, Regardless of whether or not the priority enhancement layer is aligned, a dummy reception completion packet for an unreceived transmission video packet is transmitted after the timeout period elapses from the base layer reception completion point when the base layer is aligned. It is.

  More specifically, the frame detection unit 85 is configured to try whether or not the temporary encoding is successful only for the base frame, and the timer unit 86 always starts counting from the time when the base layer reception is completed, The reception completion processing unit 84 is configured to notify the elapse of the timeout time.

  With such a second embodiment, the base frame can be protected, and real-time performance can be ensured with a minimum video quality. Furthermore, since the transmission apparatus 101 continues the retransmission process of the enhancement layer until the timeout time elapses, it is possible to increase the possibility that the enhancement layers are arranged depending on the line status. That is, the best-effort result in which the line status and the video quality are linked can also be obtained by the receiving apparatus 100 of the second embodiment.

  Other embodiments will be described.

  In the embodiment, the priority enhancement layer setting unit sets only one enhancement layer as the priority enhancement layer. However, for example, a plurality of enhancement layers may be set as the priority enhancement layer. Further, the priority enhancement layer setting unit may be configured to weight the priority of each enhancement layer according to the specified moving image characteristics.

  In the embodiment, the video data captured from the camera is sequentially scalable encoded in the video encoding unit. However, the encoded data that has been stored in the video data storage unit in advance is received. You may make it transmit to an apparatus.

  Also, if the receiving terminal is a terminal having a display resolution capability of VGA (640x480) level, such as the receiving capability of the receiving terminal such as resolution, the transmission video source resolution is a resolution of full HD (1920x1080). Even if spatial extension layer data is transmitted, it cannot be fully utilized. In addition, even if the decoding capability of the decoder that the receiver has only supports up to 720P (1280x720), even if you send the spatial extension layer data of full HD (1920x1080) resolution, it can not be fully utilized, The capability of the receiving terminal such as the maximum resolution and bit rate is exchanged in advance by negotiation between the receiving terminal and the transmitting terminal, and the transmitting terminal transmits the extension layer data in advance so that the video is less than that capability. Also good. The method described so far can make full use of its functions even when this restriction is added.

  Further, the above-described units may be installed as new programs in the existing transmission device and reception device to realize the functions of the present invention. The scalable encoding in the video encoding unit is not limited to that shown in the embodiment. For example, the enhancement layer hierarchy may be further divided into a large number.

  In addition, various modifications and combinations of embodiments may be performed without departing from the spirit of the present invention.

200: data transfer system 100: receiving device 101: transmitting device 2: encoding control unit 3: video encoding unit 4: video data storage unit 5: video characteristic specifying unit 6: priority extension layer setting unit 7: transmission processing unit 8 : Reception processing unit 9: video decoding unit 71: layer recognition unit 72: transmission unit 73: retransmission processing unit 81: reception unit 82: layer identification unit 83: FEC unit 84: reception completion processing unit 85: frame detection unit 86: timer Part

Claims (7)

In a data transfer system for transferring video data to stream type data, the receiving device receives a transmission video packet transmitted from a transmission device,
The transmission device stores both encoded video data composed of a base layer and an enhancement layer and layer data indicating a layer of the encoded video data in pairs and layer data corresponding to the encoded video data. A transmission unit that generates a stored transmission video packet and transmits the packet to the reception device, and a reception completion packet indicating the transmission video packet received by the reception device is received from the reception device. A retransmission processing unit that causes the transmission unit to retransmit a transmission video packet that has not received a packet from the receiving device;
The receiving device is
A reception completion processing unit that transmits at least a reception completion packet indicating a transmission video packet received by the reception device to the transmission device;
A layer identifying unit for identifying a layer of a transmission video packet received by the receiving device based on layer data;
A frame detection unit for detecting a base layer reception completion time point when a base layer of a reproduction target frame is aligned by a transmission video packet received by the receiving device ;
A timer unit that notifies the reception completion processing unit that a timeout period has elapsed when a predetermined timeout period has elapsed since the completion of reception of the base layer,
The reception completion processing unit is configured to transmit a reception completion packet to the transmission device with respect to an unreceived transmission video packet when notified from the timer unit that a timeout time has elapsed. Receiving device.   The receiving apparatus according to claim 1, wherein the timeout time is configured to end before the playback timing of the playback target frame. A priority enhancement layer setting unit that sets a priority enhancement layer that is an enhancement layer to be retransmitted;
The frame detection unit is configured to detect the base layer reception completion time point and a priority enhancement layer reception completion time point that is a time point when the priority enhancement layer is aligned;
The timer unit notifies the reception completion processing unit that the timeout period has elapsed when a timeout period has elapsed from the later of the base layer reception completion point and the priority enhancement layer reception completion point. The receiving apparatus according to claim 1 or 2, wherein the receiving apparatus is configured. The frame detection unit performs provisional decoding of a base layer or a priority enhancement layer using a transmission video packet received by the reception device , and the time when the provisional decoding is successful is the base layer reception completion time or the priority enhancement layer reception. The receiving device according to claim 3, which is a completion point . The reception completion processing unit obtains a sequence number smaller than the largest number among the sequence numbers of transmission video packets being received at the later time of the base layer reception completion time or the priority enhancement layer reception completion time. The receiving device according to claim 3 or 4 , wherein the receiving device is configured to transmit a reception completion packet to the transmitting device with respect to an unreceived transmission video packet. A video characteristic specifying unit for specifying the video characteristics of the video data;
The said priority enhancement layer setting part is comprised so that the priority enhancement layer which is an enhancement layer which should be retransmitted based on the video characteristic specified by the said video characteristic specific | specification part may be set. Receiver. In a data transfer system for transferring video data in a stream type, a program for a receiving device used for a receiving device that receives a transmission video packet transmitted from a transmitting device,
The transmission device stores both encoded video data composed of a base layer and an enhancement layer and layer data indicating a layer of the encoded video data in pairs and layer data corresponding to the encoded video data. A transmission unit that generates a stored transmission video packet and transmits the packet to the reception device, and a reception completion packet indicating the transmission video packet received by the reception device is received from the reception device. A retransmission processing unit that causes the transmission unit to retransmit a transmission video packet that has not received a packet from the receiving device;
The receiving device program is
A reception completion processing unit that transmits at least a reception completion packet indicating a transmission video packet received by the reception device to the transmission device;
A layer identifying unit for identifying a layer of a transmission video packet received by the receiving device based on layer data;
A frame detection unit for detecting a base layer reception completion time point when a base layer of a reproduction target frame is aligned by a transmission video packet received by the receiving device ;
A timer unit that notifies the reception completion processing unit that the time-out time has elapsed when a predetermined time-out time has elapsed since the base layer of the reproduction target frame is aligned in the frame detection unit;
The reception completion processing unit is configured to transmit a reception completion packet to the transmission device with respect to an unreceived transmission video packet when notified from the timer unit that a timeout time has elapsed. A program for a receiving device.