JP4544640B2 - Data reproducing apparatus and data reproducing method - Google Patents

Description

  The present invention relates to a data reproducing apparatus and a data reproducing method, and in particular, transmission error tolerance and video quality of image data acquired on the receiving side according to user preference and transmission error occurrence state on the receiving side of image data. Is related to data reproduction processing that enables switching between the two.

  In recent years, with the establishment of the international standard MPEG-4 (Moving Picture Experts Group, Phase 4, IS0 / IEC 14496) relating to the compression and encoding method of video and audio data, it has become possible to distribute video and audio data in a narrow band. For example, in a transmission line having a bandwidth of 64 kbit / s, an image having one screen with 176 horizontal pixels, 144 vertical pixels, and a frame rate of 5 to 6 frames / second. Data and voice data as good as telephone quality can be transmitted simultaneously.

  In the simple profile defined by the MPEG-4 video standard, I-VOP and P-VOP, which have different encoding types, are used as VOP (video object plane) which is an image of each object constituting one scene. used. Here, the I-VOP does not refer to the image data of another VOP when the image data is compressed or expanded. Therefore, the encoding process or decoding process for the I-VOP can be performed independently regardless of the image data of other VOPs. On the other hand, the P-VOP is predicted based on the image data of the I-VOP or P-VOP located immediately before the target P-VOP when the compression process or the expansion process of the image data of the P-VOP to be processed is performed. The difference component between the prediction data obtained in this way and the image data of the target P-VOP is obtained, and the difference component is encoded or decoded.

  The repetition period of I-VOP is generally set to a period in which I-VOP appears once in about 0.5 seconds in digital satellite broadcasting using a wide band. That is, in Japanese television broadcasting, the number of frames per second is about 30, so an I-VOP appears every 15 frames. On the other hand, in the narrow band, the I-VOP repetition cycle with a large code amount of the encoded image data (encoded data) is lengthened, and the P-VOP or B-VOP with a small code amount of the encoded data (that is, its) Increasing the appearance frequency of those that refer to image data of other VOPs at the time of encoding or decoding has a greater effect of improving video quality than increasing the appearance frequency of I-VOPs. However, lengthening the I-VOP repetition period, that is, lowering the frequency of occurrence of I-VOP is not preferable from the viewpoint of error resistance, and image disturbance continues for a long time when packet loss occurs. Become. Note that the VOP in MPEG-4 described above corresponds to a frame in MPEG-1,2.

  In addition, the international standardization organization 3GPP (Third Generation Partnership Project, http://www.3gpp.org), which establishes standards for receiving terminals in wireless networks, is a protocol for transmitting video data between servers and receiving terminals. A protocol for using RTP / UDP / IP (real time transport protocol / user datagram protocol / internet protocol) and requesting data from the receiving terminal to the server is RTSP / TCP / IP (real time streaming protocol / transmission). Control protocol / internet protocol) is specified. Further, in the 3GPP standard, SMIL (Synchronization Multimedia Markup Language, http://www.w3.org) can be used as a scene description language.

FIG. 18 shows a conventional data transmission system for distributing image data using the Internet.
The data transmission system 20 includes a server 20a that packetizes a video stream that is the encoded data and transmits packet data, a receiving terminal 20b that receives the video stream and reproduces image data, and the packet data. And a network 11 such as the Internet for transmission from the server 20a to the receiving terminal 20b.

  In this communication system 20, first, communication of a message Mes for making a data request to the server 20a is performed between the receiving terminal 20b and the server 20a by RTSP / TCP / IP. Request signal Dau is transmitted to server 20a. Then, the video stream Dstr is transmitted from the server 20a to the receiving terminal 20b by RTP / UDP / IP which is a data transmission protocol. In the receiving terminal 20b, the received video stream Dstr is decoded, and the image data is reproduced.

FIG. 19 is a diagram for explaining a conventional image encoding apparatus that performs an encoding process corresponding to the MPEG standard, and FIG. 19A is a block diagram showing the configuration thereof.
The image encoding apparatus 100 constitutes the server 20a shown in FIG. 18, and compresses and encodes the original image data Dv as it is when encoding I-VOP, and the original image data Dv when encoding P-VOP. The encoder 102 that compresses and encodes the difference data Dvd from the prediction data Dp and outputs the encoded data De, and the compressed data obtained by compressing the original image data Dv and the difference data Dvd by the encoder 102 A decoder 103 that decompresses Dc and compressed differential data Dcd and outputs local decoded data Dd corresponding to I-VOP and local decoded differential data Ddd corresponding to P-VOP; and the original image data Dv A subtractor 101 that creates the difference data Dvd by a subtraction process with the prediction data Dp.

  The image encoding apparatus 100 includes an adder 104 that adds the prediction data Dp to the local decoded differential data Ddd to generate local decoded data Ddp corresponding to P-VOP, and a local corresponding to the I-VOP. A frame memory 105 that records the decoded data Dd and the local decoded data Ddp corresponding to the P-VOP as reference data, and the image data read from the frame memory 105 is used as the prediction data Dp. 101 and the adder 104 are supplied.

Next, the operation of the conventional image encoding apparatus 100 will be briefly described.
In the image encoding device 100, as shown in FIG. 19B, the original image data Dv input from the outside is encoded for each VOP.
For example, the first VOP data V (1) is encoded as I-VOP, the second to fifth VOP data V (2) to V (5) are encoded as P-VOP, and the sixth The VOP data V (6) is encoded as I-VOP, and the seventh to tenth VOP data V (7) to V (10) are encoded as P-VOP.

  When the encoding process is started, first, the first VOP data V (1) is encoded as I-VOP. That is, the original image data Dv corresponding to the I-VOP is compression-encoded by the encoder 102 and output as encoded data De. At this time, the encoder 102 outputs the compressed data Dc obtained by compressing the original image data Dv to the decoder 103. Then, the decoder 103 performs decompression processing on the compressed data Dc to generate I-VOP local decoded data Dd. The local decoded data Dd output from the decoder 103 is stored in the frame memory 105 as reference data.

  Next, the second VOP data V (2) is encoded as P-VOP. That is, the original image data Dv corresponding to the P-VOP is input to the subtracter 101 in the preceding stage of the encoder 102, and the subtractor 101 reads the image data read out as the predicted data Dp from the frame memory 105, and Difference data Dvd from the original image data Dv corresponding to the P-VOP is generated. Then, the difference data Dvd is compression-encoded by the encoder 102 and output as encoded data De.

  At this time, the encoder 102 outputs the compressed differential data Dcd obtained by compressing the differential data Dvd to the decoder 103. Then, the decoder 103 performs decompression processing on the compressed differential data Dcd to generate local decoded differential data Ddd. The adder 104 corresponds to P-VOP by adding the locally decoded differential data Ddd output from the decoder 103 and the prediction data Dp that is image data read from the frame memory 105. Local decoded data Ddp is generated. The locally decoded data Ddp output from the adder 104 is stored in the frame memory 105 as reference data.

Thereafter, the third to fifth VOP data V (3) to V (5) are encoded as P-VOP, similar to the second VOP data. Further, the sixth VOP data V (6) is encoded as I-VOP, similar to the first VOP data V (1), and the seventh to tenth VOP data V ( 7)) to V (10) are encoded as P-VOP, similarly to the second VOP data V (2).
As described above, in the image encoding apparatus 100, the encoding process for the original image data Dv is performed with the I-VOP cycle being 5 VOPs.

FIG. 20 is a block diagram for explaining a conventional image decoding apparatus.
The image decoding apparatus 200 decodes the encoded data De output from the image encoding apparatus 100 shown in FIG. 19A, and constitutes a decoding unit of the receiving terminal 20b in the data transmission system 20. To do.
That is, the image decoding apparatus 200 performs decompression decoding processing on the encoded data De from the image encoding apparatus 100 in units of VOPs, and decodes the decoded data Dd corresponding to the original image data Dv when decoding the I-VOP. When decoding the P-VOP, the decoder 201 that outputs the decoded difference data Ddd corresponding to the difference data Dvd between the original image data Dv and the prediction data Dp, and the decoded difference data Ddd Refer to the adder 202 that adds the prediction data Dp to generate the decoded data Ddecp corresponding to the P-VOP, the decoded data Dd corresponding to the I-VOP, and the decoded data Ddecp corresponding to the P-VOP. A frame memory 203 for recording as data, and image data read out from the frame memory 203 as the prediction data Dp is It is intended to be supplied to the adder 202.

Next, the operation of the conventional image decoding apparatus 200 will be briefly described.
When the decoding process is started, the image decoding apparatus 200 decodes the encoded data De from the image encoding apparatus 100 for each VOP.

  That is, when encoded data De corresponding to I-VOP is input to the decoder 201, the decoder 201 performs decompression decoding on the encoded data De and corresponds to the original image data Dv. The decrypted data Dd to be generated is generated. The decoded data Dd is output from the image decoding apparatus 200 and stored in the frame memory 203 as reference data.

  Also, when the encoded data De corresponding to the P-VOP is input to the decoder 201, the decoder 201 performs decompression decoding on the encoded data De to obtain the original image data Dv and its image data Dv. Decoded differential data Ddd corresponding to differential data Dvd with the prediction data Dp is generated. When the decoded differential data Ddd is input to the adder 202, the adder 202 adds the decoded differential data Ddd and the image data read out as the predicted data Dp from the frame memory 203. Processing is performed to generate decoded data Ddecp corresponding to the P-VOP. The decoded data Ddecp is output from the image decoding apparatus 200 and stored in the frame memory 203 as reference data.

However, the conventional data transmission system 20 as shown in FIG. 18 has the following problems.
That is, in data transmission using RTP / UDP / IP, data sent from the distribution server may not arrive at the receiving terminal due to the characteristics of the protocol. One of the factors is that when a bit error occurs in the received packet, the incoming packet is discarded by the error detection mechanism in UDP. In particular, in a transmission system in which a wireless transmission path is included in the transmission path from the server to the receiving terminal, the received transmission data cannot be correctly demodulated when the radio field intensity at the receiving terminal is weak. Bit errors will occur.

  In addition, the receiving terminal cannot perform the decoding process on the video frame unless the data (video stream) for one frame (VOP) is prepared. For this reason, as a countermeasure method when a transmission error occurs, for example, when a transmission error occurs, the data of the frame (VOP) in which data was not normally received is discarded, and then the I frame (I-VOP) A method is used in which a video frame in which data has been normally received is displayed until the data is normally received, and when the data in the I frame is normally received, the decoding process is resumed from this I frame. . With this handling method, there is no video disturbance, but the movement of the display image stops until an I frame is received.

  Furthermore, as another method when a transmission error occurs, the data of the frame (VOP) in which the data was not normally received is replaced with the data of the frame (VOP) that has been correctly received and decoded immediately before. There is a method of using frame data for decoding of subsequent frames. In this method, the display image does not stop moving in a frame other than the frame in which data is not normally received, so that smooth display is performed. However, since the data of the target frame to be decoded is decoded with reference to a frame different from the frame referred to in the encoding process, there is a possibility that the display content is greatly disturbed. Although depending on the viewer's preference, in general, when a transmission error occurs, rather than using a method of replacing the discarded reference frame data for the target frame with data of a frame other than the reference frame, By using the method of displaying the frame immediately before the occurrence of the transmission error until the I frame data is normally received after the occurrence of the transmission error, a reproduced image with less sense of incongruity can be obtained.

However, the conventional receiving terminal is set in advance to execute any one of the above methods as a response method when a transmission error occurs. For this reason, an image displayed when a transmission error occurs is displayed. On the other hand, there was a problem that viewers sometimes felt a great sense of incongruity.
Furthermore, in order to suppress the degradation of video quality due to data compression, the appearance frequency of I frames (I-VOP) should be reduced as much as possible, but on the other hand, decoding that has become abnormal due to the occurrence of a transmission error From the standpoint of quickly returning the normalization process to the normal decoding process, there is also a problem that the frequency of appearance of the I frame (I-VOP) cannot be reduced very much.

  The present invention has been made in order to solve the above-described problems, and is a data reproducing apparatus and data that can make an image displayed when a transmission error occurs almost completely uncomfortable for a viewer. It is an object of the present invention to obtain a reproduction method, a program for performing the data reproduction method by software, and a data recording medium storing the program.

  A data reproduction device according to the present invention is a data reproduction device that receives and reproduces any one of a plurality of pieces of image data having different error tolerances, and is auxiliary data indicating error tolerance strength of the plurality of image data That specifies one of the plurality of image data based on the auxiliary data receiving unit that receives the image data, the condition relating to the image data to be received, and the error tolerance strength of each of the image data indicated by the auxiliary data A data designation unit that generates a designation signal, a data transmission unit that transmits the data designation signal, and the transmitted image data selected from the plurality of image data based on the transmitted data designation signal And an image data receiving unit.

  According to the present invention, in the data reproduction apparatus, each of the plurality of pieces of image data having different error tolerances is obtained by encoding an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and a digital video signal. And the inter-screen encoded data encoded using the inter-screen pixel value correlation, and the application intervals of the intra-screen encoded data in the respective image data are different.

  According to the present invention, in the data reproduction apparatus, the plurality of image data having different error tolerances are first and second image encoded data obtained by encoding a digital video signal, The encoded image data is data in which encoded data corresponding to one frame is packetized for each data unit smaller than the frame, and the second encoded image data is encoded data corresponding to one frame. The packetization is performed for each frame or for each data unit larger than the frame.

  The present invention is characterized in that, in the data reproducing apparatus, the plurality of pieces of image data having different error tolerances correspond to the same image series and have different frame rates. It is.

  The present invention is characterized in that, in the data reproduction apparatus, the plurality of pieces of image data having different error tolerances correspond to the same image series and have different transmission protocols for the pieces of image data. It is what.

  The present invention provides a decoding unit that decodes image data received by the image data receiving unit and a control for switching an operation mode of the decoding unit according to a set operation condition in the data reproduction device. And a section.

  According to the present invention, in the data reproduction apparatus, each of the plurality of pieces of image data having different error tolerances is obtained by encoding an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and a digital video signal. Inter-screen encoded data encoded using inter-screen pixel value correlation, and the appearance interval of the intra-screen encoded data in each of the image data is different. When the transmission condition is an operation condition for receiving the image data in which the appearance interval of the intra-screen encoded data is shorter than the default value, the operation mode of the decoding unit is changed to the normal intra-screen encoded data after that. The first decoding mode in which the decoding process is temporarily stopped until reception is performed, and the operation condition is an operation condition for receiving image data in which the appearance interval of the in-screen encoded data is equal to or greater than a predetermined value. When the transmission error occurs, the inter-screen encoded data cannot be decoded due to the transmission error until the intra-screen encoded data is normally received after that. In this case, the second decoding mode for decoding is performed except for the above-described part.

  The present invention is characterized in that, in the data reproducing apparatus, the second decoding mode is for decoding image data of a frame other than a frame in which data is lost due to occurrence of a transmission error.

  In the data reproduction apparatus, the image data is packetized for each data unit smaller than a frame, and the second decoding mode is other than a packet in which data is lost due to occurrence of a transmission error. The image data of the packet is decoded.

  A data reproduction device according to the present invention is a data reproduction device that receives and reproduces any one of a plurality of pieces of image data having different error tolerances, and is auxiliary data indicating error tolerance strength of the plurality of image data An auxiliary data receiving unit that receives the error, an error detection unit that detects an error occurrence rate of the received image data, an error occurrence rate of the detected image data, and an error tolerance strength of each image data indicated by the auxiliary data A data specifying unit for generating a data specifying signal for specifying one of the plurality of image data, a data transmitting unit for transmitting the data specifying signal, and the plurality of data based on the transmitted data specifying signal. And an image data receiving unit that receives the transmitted image data selected from the image data.

  A data reproduction method according to the present invention is a data reproduction method for receiving and reproducing any one of a plurality of pieces of image data having different error tolerances, the auxiliary data indicating the error tolerance strength of the plurality of image data For specifying one of the plurality of image data based on the auxiliary data receiving step for receiving the image data, the condition relating to the image data to be received, and the error tolerance strength of each image data indicated by the auxiliary data A data designation step for generating a designation signal; a data transmission step for transmitting the data designation signal; and a selection of the plurality of image data based on the transmitted data designation signal, and reception of the transmitted image data And an image data receiving step.

  A data reproduction method according to the present invention is a data reproduction method for receiving and reproducing any one of a plurality of pieces of image data having different error tolerances, the auxiliary data indicating the error tolerance strength of the plurality of image data Auxiliary data receiving step, an error detecting step of detecting an error occurrence rate of the received image data, an error occurrence rate of the detected image data, and an error tolerance strength of each image data indicated by the auxiliary data Based on the data designation step for generating a data designation signal for designating one of the plurality of image data, a data transmission step for transmitting the data designation signal, and on the basis of the transmitted data designation signal An image data receiving step for receiving the transmitted image data selected from the plurality of image data. It is intended to.

  A data recording medium according to the present invention is a data recording medium storing a data reproduction program for performing data reproduction processing by a computer for receiving and reproducing any of a plurality of pieces of image data having different error tolerances. The data reproduction program includes an auxiliary data receiving step for receiving auxiliary data indicating error tolerance strength of the plurality of image data, a condition relating to image data to be received, and an error of each image data indicated by the auxiliary data. A data designation step for generating a data designation signal for designating one of the plurality of image data based on the tolerance strength, a data transmission step for transmitting the data designation signal, and the transmitted data designation signal Based on the image data selected from the plurality of image data and receiving the transmitted image data. It is characterized in that the invention will include a data receiving step.

  A data recording medium according to the present invention is a data recording medium storing a data reproduction program for performing data reproduction processing by a computer for receiving and reproducing any of a plurality of pieces of image data having different error tolerances. The data reproduction program includes an auxiliary data receiving step for receiving auxiliary data indicating error tolerance strength of the plurality of image data, an error detecting step for detecting an error occurrence rate of the received image data, and the detected image. A data designating step for generating a data designating signal for designating one of the plurality of image data based on the error rate of data and the error resilience strength of each image data indicated by the auxiliary data; A data transmission step of transmitting a data designation signal, and the plurality of data based on the transmitted data designation signal Is selected from among the image data, it is characterized in that the invention will include an image data reception step of receiving the transmitted image data.

  A data reproduction apparatus according to the present invention is a data reproduction apparatus that receives and reproduces image data, the image data receiving unit receiving the image data, and decoding the image data received by the image data receiving unit And a control unit that switches an operation mode of the decoding unit according to a set condition.

  The present invention is characterized in that the data reproducing apparatus includes an auxiliary data receiving unit that receives auxiliary data indicating the error tolerance strength of the image data.

  In the data reproducing apparatus according to the present invention, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-pixel value correlation, and a digital video signal using inter-screen pixel value correlation. The inter-coded data is encoded, and the auxiliary data represents an appearance interval of the intra-coded data in the image data.

  In the data reproducing apparatus according to the present invention, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-pixel value correlation, and a digital video signal using inter-screen pixel value correlation. Encoded image data, and the image data receiving unit includes an operation unit that calculates an appearance interval of the encoded image data in the image data. Is.

  In the data reproducing apparatus according to the present invention, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-pixel value correlation, and a digital video signal using inter-screen pixel value correlation. The inter-coded data encoded, and the control unit receives the image data when the appearance interval of the intra-coded data is shorter than a predetermined value indicated by the setting condition. The operation mode of the encoding unit is set to a first decoding mode in which the decoding process is temporarily stopped until transmission of the encoded data within the screen is normally received when a transmission error occurs. However, when receiving image data that exceeds the default value indicated by the setting condition, the operation mode of the decoding unit is the part that cannot be decoded due to a transmission error when a transmission error occurs. It is characterized in that it is an second decoding mode for decoding the exception.

  The present invention is characterized in that, in the data reproducing apparatus, the second decoding mode is for decoding image data of a frame other than a frame in which data is lost due to occurrence of a transmission error.

  In the data reproduction apparatus, the image data is packetized for each data unit smaller than a frame, and the second decoding mode is other than a packet in which data is lost due to occurrence of a transmission error. The image data of the packet is decoded.

  In the data reproducing apparatus according to the present invention, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-pixel value correlation, and a digital video signal using inter-screen pixel value correlation. Encoded inter-screen encoded data, and the control unit performs the decoding operation of the decoding unit when a transmission error occurs, the decoding time of the frame in which the transmission error has occurred, Switching is performed in accordance with the time difference from the decoding time of the intra-frame encoded frame decoded thereafter.

  The present invention provides the data reproducing apparatus, wherein, when a transmission error occurs, the control unit performs a time difference from the time of decoding the frame in which the transmission error has occurred to the time of decoding of the intra-frame encoded frame to be decoded thereafter. In the first case where the set condition is smaller than a certain reference value, the decoding unit performs the decoding operation on the intra-frame encoded frame after decoding the frame in which the transmission error has occurred. In the meantime, the decoding process for the image data is controlled to stop, and the time difference from the decoding of the frame in which the transmission error has occurred to the decoding of the intra-frame encoded frame to be decoded thereafter is In the second case where the set condition is equal to or greater than a certain reference value, the decoding operation of the decoding unit is performed after the decoding of the frame in which the transmission error occurs. Until the decoding of the intra-frame encoded frame is performed, control is performed so as to perform a decoding process for decoding the inter-frame encoded data except for the portion that cannot be decoded due to the transmission error. It is characterized by this.

  In the data reproducing apparatus according to the present invention, the decoding process performed in the second case is to decode image data of a frame other than a frame in which data is lost due to a transmission error. Is.

  According to the present invention, in the data reproduction apparatus, the image data is packetized for each data unit smaller than a frame, and the decoding process performed in the second case is lost due to a transmission error. The image data of packets other than the received packet is decoded.

  A data reproduction method according to the present invention is a data reproduction method for receiving and reproducing image data, wherein the image data receiving step for receiving the image data and the image data received by the image data receiving unit are decoded. And a control step of switching the operation mode of the decoding unit in accordance with the set condition.

  A data recording medium according to the present invention is a data recording medium storing a data reproduction program for performing data reproduction processing for receiving and reproducing image data by a computer, and the data reproduction program receives the image data. An image data receiving step, a decoding step for decoding the image data received by the image data receiving unit, and a control step for switching the operation mode of the decoding unit according to a set condition. It is characterized by that.

  A data reproduction apparatus according to the present invention is a data reproduction apparatus that switches and displays a plurality of video streams, receives an RTP packet constituting the video stream, extracts an RTP time stamp from the RTP packet, and extracts the RTP A receiving unit that calculates a display time on the data reproduction device corresponding to the time stamp, an error detecting unit that detects an error from an RTP packet constituting the first video stream being received, and the error detecting unit A transmission / reception unit that requests transmission of the second video stream when an error is detected, an analysis unit that analyzes the display time calculated by the reception unit, and a control unit that switches the video stream to be displayed, The reception unit receives the first video stream after a transmission request for the second video stream is made. Receiving the RTP packet constituting the stream and the RTP packet constituting the second video stream, and determining the first display time from the RTP timestamp of the first video stream, and the second video stream A second display time is calculated from the RTP time stamp, and the analysis unit compares the first display time with the second display time, and the second display time is the first display time. When the second display time is larger than the first display time, the control unit switches the video stream to be displayed to the second video stream.

  The present invention is characterized in that, in the data reproduction device, the transmission / reception unit requests to stop transmission of the first video stream when the second display time is larger than the first display time.

  According to the present invention, in the data reproduction device, the receiving unit stops receiving the RTP packets constituting the first video stream when the second display time is larger than the first display time. Features.

  According to the present invention, in the data reproduction device, the receiving unit configures the first video stream after the second display time when the second display time is larger than the first display time. The RTP packet is discarded.

  A data reproduction method according to the present invention is a data reproduction method for switching and displaying a plurality of video streams, the step of receiving RTP packets constituting the video stream, and the construction of the first video stream being received. Detecting an error from an RTP packet to be transmitted, a step of requesting transmission of a second video stream when the error is detected, and a request for transmitting the second video stream, Receiving an RTP packet constituting one video stream and an RTP packet constituting the second video stream; extracting an RTP timestamp from the RTP packet constituting the first video stream; The first display time on the data reproduction device corresponding to the RTP time stamp is Extracting a RTP time stamp from an RTP packet constituting the second video stream and calculating a second display time on the data reproduction device corresponding to the RTP time stamp; Comparing the first display time with the second display time to determine whether the second display time is greater than the first display time; and And a step of switching the video stream to be displayed to the second video stream when the display time is larger than one display time.

  A data reproduction program according to the present invention is a data reproduction program for performing data reproduction processing for receiving and displaying a plurality of video streams by a computer, the step of receiving RTP packets constituting the video stream, and reception A step of detecting an error from an RTP packet constituting the first video stream, a step of requesting transmission of the second video stream when the error is detected, and a step of transmitting the second video stream Receiving a RTP packet constituting the first video stream and an RTP packet constituting the second video stream after a transmission request is made; and an RTP packet constituting the first video stream. The RTP timestamp is extracted from the RTP timestamp Calculating a first display time on the data reproduction apparatus corresponding to the RTP time, extracting an RTP time stamp from an RTP packet constituting the second video stream, and reproducing the data corresponding to the RTP time stamp Comparing the step of calculating the second display time on the device with the first display time and the second display time, whether or not the second display time is greater than the first display time And a step of switching a video stream to be displayed to the second video stream when the second display time is greater than the first display time.

  A data recording medium according to the present invention is a recording medium for storing a data reproduction program for performing a data reproduction process for receiving and displaying a plurality of video streams by a computer, and the data reproduction program stores the video streams. A step of receiving a constituent RTP packet, a step of detecting an error from an RTP packet constituting the first video stream being received, and a transmission request for the second video stream when the error is detected. Performing, after receiving a transmission request for the second video stream, receiving an RTP packet constituting the first video stream and an RTP packet constituting the second video stream; From the RTP packet constituting the first video stream to the RTP tag A first display time corresponding to the RTP time stamp is calculated on the data reproduction apparatus, and an RTP time stamp is extracted from an RTP packet constituting the second video stream. The step of calculating the second display time on the data reproduction device corresponding to the RTP time stamp is compared with the first display time and the second display time, and the second display time is Determining whether the display time is greater than the first display time and switching the video stream to be displayed to the second video stream when the second display time is greater than the first display time. It is characterized by that.

  The present invention is characterized in that the data reproducing apparatus is a mobile phone.

  As described above, according to the data reproducing apparatus of the present invention, a data reproducing apparatus that receives and reproduces any one of a plurality of pieces of image data having different error tolerances, Based on the auxiliary data receiving unit that receives the auxiliary data indicating the error tolerance strength, the condition regarding the image data to be received, and the error tolerance strength of each of the image data indicated by the auxiliary data, among the plurality of image data A data specifying unit that generates one of the data specifying signal, a data transmitting unit that transmits the data specifying signal, and the plurality of image data selected based on the transmitted data specifying signal, An image data receiving unit for receiving transmitted image data is provided, so that the user has a discomfort with the display image when a transmission error occurs So that the smaller, it is possible to select the video stream to be provided to the receiving terminal.

  According to the present invention, in the data reproduction apparatus, each of the plurality of pieces of image data having different error tolerances includes an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and a digital video. Video signal encoded with inter-screen pixel value correlation, and the appearance interval of the intra-screen encoded data in each image data is different. By selecting the appearance interval of the I frame in the stream, it is possible to select image data having error tolerance according to the transmission error occurrence status and user preference.

  According to the present invention, in the data reproduction device, the plurality of pieces of image data having different error tolerances are first and second image encoded data obtained by encoding a digital video signal. The encoded image data of 1 is that the encoded data corresponding to one frame is packetized for each data unit smaller than the frame, and the second encoded image data is encoded corresponding to one frame. Since the data is packetized for each frame or for each data unit larger than the frame, by selecting the packet size of the encoded image data, a video stream with high transmission error resistance and video It is possible to switch between high quality video streams.

  According to the present invention, in the data reproduction device, the plurality of pieces of image data having different error tolerances correspond to the same image series and have different frame rates. Therefore, by selecting the frame rate, it is possible to switch between a video stream with high transmission error resistance and a video stream with good video quality.

  According to the present invention, in the data reproduction apparatus, the plurality of pieces of image data having different error tolerances correspond to the same image series and have different transmission protocols for the pieces of image data. Therefore, it is possible to select image data having different error tolerance depending on the selection of the protocol.

  According to the present invention, in the data reproduction device, the decoding unit that decodes the image data received by the image data receiving unit, and the operation mode of the decoding unit according to the set operation condition. And a control unit for switching, so that a sense of incongruity of the display image when an error occurs can be reduced by setting operation conditions.

  According to the present invention, in the data reproduction apparatus, each of the plurality of pieces of image data having different error tolerances includes an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and a digital video. Including inter-screen encoded data obtained by encoding a signal using inter-pixel value correlation, and the appearance interval of the intra-screen encoded data in each of the image data is different. When the condition is an operation condition for receiving image data in which the appearance interval of the intra-coded data is shorter than a predetermined value, the operation mode of the decoding unit is changed to the intra-coded data after that when a transmission error occurs. The first decoding mode in which the decoding process is temporarily stopped until it is normally received, and the operation condition receives image data in which the appearance interval of the in-screen encoded data is equal to or larger than a predetermined value. When the operation conditions are satisfied, the operation mode of the decoding unit is set. When a transmission error occurs, the inter-coded data cannot be decoded due to the transmission error until the intra-coded data is received normally. Since it is the second decoding mode that performs decoding except for the part that has become an error, the decoding operation at the time of error occurrence has a little discomfort in the display image according to the setting of the operating condition And can.

  According to the present invention, in the data reproduction device, the second decoding mode is characterized in that image data of a frame other than a frame in which data is lost due to the occurrence of a transmission error is decoded. Even when a transmission error occurs, it is possible to display a frame other than a frame in which data is lost, and a smooth image display can be performed.

  According to the present invention, in the data reproducing apparatus, the image data is packetized for each data unit smaller than a frame, and the second decoding mode is a packet in which data is lost due to a transmission error. It is characterized by decoding the image data of packets other than that. When a transmission error occurs, even if the frame is missing due to the error, a part of the frame can be displayed, making it smoother. Image display is possible.

  According to the data reproducing apparatus of the present invention, a data reproducing apparatus that receives and reproduces any one of a plurality of pieces of image data having different error tolerances, and indicates the error tolerance strength of the plurality of image data. An auxiliary data receiving unit that receives auxiliary data, an error detection unit that detects an error occurrence rate of received image data, an error occurrence rate of the detected image data, and an error tolerance strength of each image data indicated by the auxiliary data Based on the data designation unit for generating a data designation signal for designating one of the plurality of image data, a data transmission unit for transmitting the data designation signal, and on the basis of the transmitted data designation signal And an image data receiving unit that receives the transmitted image data selected from the plurality of image data. It can be changed according to the incidence of transmission error occurrence interval of I-frame in the stream, which makes it possible to reduce the uncomfortable feeling of the displayed image at the time of occurrence of transmission errors.

  According to the data reproduction method of the present invention, a data reproduction method for receiving and reproducing any one of a plurality of pieces of image data having different error tolerances, the error tolerance strength of the plurality of image data being indicated. One of the plurality of image data is designated based on the auxiliary data receiving step for receiving the auxiliary data, the condition regarding the image data to be received, and the error tolerance strength of each of the image data indicated by the auxiliary data A data designation step for generating a data designation signal to be transmitted, a data transmission step for transmitting the data designation signal, and image data selected and transmitted from the plurality of image data based on the transmitted data designation signal Image data receiving step for receiving the image data, so that the user can display the display image when a transmission error occurs. Ku so that uncomfortable feeling is reduced, it is possible to select the video stream to be provided to the receiving terminal, this makes it possible to reduce the uncomfortable feeling of the displayed image at the time of occurrence of transmission errors.

  According to the data reproduction method of the present invention, a data reproduction method for receiving and reproducing any one of a plurality of pieces of image data having different error tolerances, the error tolerance strength of the plurality of image data being indicated. An auxiliary data receiving step for receiving auxiliary data, an error detecting step for detecting an error occurrence rate of the received image data, an error occurrence rate of the detected image data, and error tolerance of each image data indicated by the auxiliary data A data designating step for generating a data designating signal designating one of the plurality of image data based on the intensity, a data sending step for sending the data designating signal, and the transmitted data designating signal An image data receiving step of receiving the transmitted image data selected from the plurality of image data based on the image data Because characterized, for example, the appearance interval of the I-frame in the video stream can be changed in accordance with the incidence of transmission error, which makes it possible to reduce the uncomfortable feeling of the displayed image at the time of occurrence of transmission errors.

  According to the data recording medium of the present invention, a data recording medium storing a data reproduction program for performing data reproduction processing for receiving and reproducing any one of a plurality of pieces of image data having different error tolerances. The data reproduction program includes an auxiliary data receiving step for receiving auxiliary data indicating error tolerance strength of the plurality of image data, a condition relating to image data to be received, and each image data indicated by the auxiliary data. A data designating step for generating a data designating signal designating one of the plurality of image data based on the error tolerance strength of the data, a data sending step for sending the data designating signal, and the transmitted data Receives the transmitted image data selected from the plurality of image data based on the designated signal The image data receiving step is included, so that it is possible to realize a decoding process that can suppress the uncomfortable feeling of the display image when a transmission error occurs by software by a user setting at the receiving terminal. Become.

  According to the data recording medium of the present invention, a data recording medium storing a data reproduction program for performing data reproduction processing for receiving and reproducing any one of a plurality of pieces of image data having different error tolerances. The data reproduction program includes an auxiliary data receiving step for receiving auxiliary data indicating error tolerance strength of the plurality of image data, an error detecting step for detecting an error occurrence rate of the received image data, and the detection A data designation step for generating a data designation signal for designating one of the plurality of image data based on the error occurrence rate of the image data and the error resilience strength of each image data indicated by the auxiliary data; A data transmission step for transmitting the data designation signal, and an operation based on the transmitted data designation signal. An image data receiving step for receiving transmitted image data selected from a plurality of image data, and, for example, changing an I-frame appearance interval according to a transmission error occurrence rate Thus, a decoding process that can suppress a sense of incongruity of a display image when a transmission error occurs can be realized by software.

  According to the data reproducing apparatus of the present invention, a data reproducing apparatus that receives and reproduces image data, the image data receiving unit that receives the image data, and the image data received by the image data receiving unit And a control unit that switches the operation mode of the decoding unit according to a set condition. When an error occurs, the operation mode of the decoding unit is set. Thus, the display image can be changed to one with less discomfort.

  According to the present invention, the data reproduction apparatus includes an auxiliary data receiving unit that receives auxiliary data indicating the error resilience strength of the image data. Therefore, when an error occurs, an operation mode of the decoding unit is provided. Can be changed to a display image with less discomfort based on the error tolerance strength of the image data being received.

  According to the present invention, in the data reproduction apparatus, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and an inter-screen pixel value correlation between the digital video signal and the inter-screen pixel value correlation. And the inter-coded data obtained by encoding, and the auxiliary data indicates the appearance interval of the intra-coded data in the image data. The operation mode of the decoding unit can be changed to a display image with less discomfort based on the appearance interval of the intra-screen encoded data in the received image data.

  According to the present invention, in the data reproducing apparatus, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-pixel value correlation, and an inter-screen pixel value correlation between the digital video signal and the inter-screen pixel value correlation. Inter-coded data obtained by encoding the image data, and the image data receiving unit includes a calculation unit that calculates an appearance interval of the intra-coded data in the image data. Therefore, when an error occurs, the operation mode of the decoding unit can be changed to a display image with less discomfort based on the appearance interval of the intra-screen encoded data in the image data being received.

  According to the present invention, in the data reproduction apparatus, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and an inter-screen pixel value correlation between the digital video signal and the inter-screen pixel value correlation. Inter-screen encoded data that is encoded using, and when the control unit receives image data in which the appearance interval of the intra-screen encoded data is shorter than a predetermined value indicated by the setting condition, The operation mode of the decoding unit is set to a first decoding mode in which the decoding process is temporarily stopped until a transmission error occurs and then the encoded data within the screen is normally received. When receiving image data whose appearance interval is equal to or greater than the predetermined value indicated by the setting condition, the decoding unit cannot be decoded due to a transmission error when a transmission error occurs. Since it is the second decoding mode that performs decoding except for the part, the decoding operation at the time of the error occurrence is made to have a less uncomfortable display image according to the setting conditions on the receiving terminal side. it can.

  According to the present invention, in the data reproduction device, the second decoding mode is characterized in that image data of a frame other than a frame in which data is lost due to the occurrence of a transmission error is decoded. Even when a transmission error occurs, it is possible to display a frame other than a frame in which data is lost, and a smooth image display can be performed.

  According to the present invention, in the data reproducing apparatus, the image data is packetized for each data unit smaller than a frame, and the second decoding mode is a packet in which data is lost due to a transmission error. It is characterized by decoding the image data of packets other than that. When a transmission error occurs, even if the frame is missing due to the error, a part of the frame can be displayed, making it smoother. Image display is possible.

  According to the present invention, in the data reproduction apparatus, the image data includes an intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation, and an inter-screen pixel value correlation between the digital video signal and the inter-screen pixel value correlation. And the inter-coded data encoded by the control unit. The control unit performs the decoding operation of the decoding unit when a transmission error occurs, and the decoding time of the frame in which the transmission error has occurred. And the subsequent decoding time of the intra-frame encoded frame to be decoded are switched according to the time difference between the decoding time of the image displayed when the transmission error occurs. It can be made with less discomfort.

  According to the present invention, in the data reproduction apparatus, when a transmission error occurs, the control unit performs from decoding of the frame in which the transmission error has occurred to decoding of an intra-frame encoded frame to be decoded thereafter. In the first case where the time difference is smaller than a certain reference value indicated by the set condition, the decoding operation of the decoding unit is performed with respect to the intra-coded frame after decoding the frame in which the transmission error has occurred. Until decoding is performed, the decoding process for the image data is controlled to stop, and the time difference between the decoding of the frame in which the transmission error has occurred and the decoding of the intra-frame encoded frame to be decoded thereafter is In the second case where the set condition is equal to or greater than a certain reference value, the decoding unit performs the decoding operation from the time of decoding the frame in which the transmission error has occurred. Until the decoding of the intra-frame encoded frame is performed after this, the inter-frame encoded data is controlled so as to be decoded except for the portion that cannot be decoded due to the transmission error. Therefore, when a transmission error occurs, it is possible to perform a display with less discomfort between a display without image disturbance and a smooth display of movement.

  According to the present invention, in the data reproduction apparatus, the decoding process performed in the second case is to decode image data of a frame other than a frame in which data is lost due to a transmission error. Therefore, if the time from the occurrence of a transmission error until the subsequent decoding of the intra-frame encoded frame is relatively long, the decoding operation at the time of transmission error occurrence is performed for a frame other than a frame with missing data. A frame can be displayed and a smooth image can be displayed.

  According to the present invention, in the data reproducing apparatus, the image data is packetized for each data unit smaller than a frame, and the decoding process performed in the second case is performed by generating a transmission error. When the image data of a packet other than a packet with a missing packet is decoded, the time from the occurrence of a transmission error to the subsequent decoding of the intra-frame encoded frame is relatively long In this case, the decoding operation when a transmission error occurs can display a part of the frame even if data is lost due to the occurrence of the error, thereby enabling smoother image display.

  According to the data reproducing method of the present invention, a data reproducing method for receiving and reproducing image data, the image data receiving step for receiving the image data, and the image data received by the image data receiving unit And a control step of switching the operation mode of the decoding unit according to the set conditions.When an error occurs, the operation mode of the decoding unit is The display image can be changed to one with less discomfort.

  The data recording medium according to the present invention is a data recording medium storing a data reproduction program for performing data reproduction processing for receiving and reproducing image data by a computer, wherein the data reproduction program includes the image data An image data receiving step for receiving the image data, a decoding step for decoding the image data received by the image data receiving unit, and a control step for switching the operation mode of the decoding unit according to a set condition; Therefore, when an error occurs, a decoding process for changing the decoding mode to a display image with less discomfort can be realized by software.

Embodiments of the present invention will be described below.
(Embodiment 1)
FIG. 1 is a diagram for explaining a data transmission system according to Embodiment 1 of the present invention. FIG. 1 (a) shows the configuration of the system, and FIG. 1 (b) shows data transmission processing in the system. Is shown.
The data transmission system 10a according to the first embodiment includes a server 100a that transmits a predetermined video stream (encoded image data), and a receiving terminal that receives the video stream transmitted from the server 100a and reproduces video data. Client terminal) 200a and a network 11 for transmitting the video stream from the server 100a to the receiving terminal 200a.

  Here, the server 100a stores a plurality of video streams obtained by encoding digital video signals of the same image sequence under different encoding conditions, and SMIL data in which attributes of the video streams are described. The data storage unit 120 stores the data stored in the data storage unit 120 and the data transmission unit 110 a that transmits the data stored in the data storage unit 120 to the network 11. The data storage unit 120 uses a mass storage device such as a hard disk.

In the first embodiment, the plurality of video streams are a plurality of image data having different error tolerances corresponding to the same image series. Specifically, each of the plurality of video streams uses a large amount of intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation and a digital video signal using inter-screen pixel value correlation. The inter-frame encoded data with a small code amount is encoded, and the appearance interval of the intra-frame encoded data in each image data, in other words, the cycle of the I frame (I-VOP) is different.
The data storage unit 120 such as the hard disk stores video streams having different I frame periods, that is, I frame periods of 10 seconds, 5 seconds, 2 seconds, and 1 second, as video files Dv1 to Dv4. The SMIL file FSD1 is stored as the SMIL data Da.

FIG. 2A shows the description content of the SMIL file FSD1.
Character strings such as <smil>, </ smil>, <body>, </ body>, <switch>, </ switch>, <video> described at the beginning of each line of the SMIL file FSD1 are elements (elements) ) And declares the contents of the description following the element.
For example, the smil element 710a and the / smil element 710b declare that the line located between the line including the smil element and the line including the / smil element is described according to the SMIL standard.
The body element 720a and the / body element 720b are an attribute of video data to be played back, for example, information (URL) indicating a location, code, and the like in a line located between the line including the body element and the line including the / body element. It declares that information related to the initialization parameter (I frame period) is described.

The switch element 730a and the / switch element 730b declare that one of a plurality of video elements positioned between the line including the switch element and the line including the / switch element is to be selected. is there. The video element declares that moving image data is designated by the description of lines 701 to 704 including the video element.
For example, in the item of each video element in the SMIL file FSD1, the appearance interval of the I frame (I frame period) is described as an i-frame-interval attribute. The best matching video element is selected. Specific values of the i-frame-interval attribute include “1s”, “2s”, “5s”, “10s”, and the smaller the specific i-frame-interval attribute value is, the more error the video data file has. It has high resistance strength. Here, four video data files having different I-frame appearance intervals are shown, but it goes without saying that it may be two, three, or five or more.

In addition, the attribute value included in each video element item is not limited to the i-frame-interval attribute, and may be a system-error-resilient-level attribute that directly indicates error resilience strength.
For example, FIG. 5 (a) shows a SMIL file FSD2 indicating four video data files having different error resilience strengths as another example of the SMIL file.

  The SMIL file FSD2 includes items related to four video elements 711 to 714 having different error resilience strength described between a line including the switch element 731a and a line including the / switch element 731b. In addition, in each video element item, error resilience strength is described as a system-error-resilient-level attribute. Based on this attribute, the video element that best matches the content of the user setting is selected.

  Here, the specific values of the system-error-resilient-level attribute in each of the video elements 711, 712, 713, 714 are “1”, “2”, “3”, and “4”, respectively.

FIG. 3 is a diagram showing a detailed configuration of the server 100a and the client terminal 200a constituting the system.
The data transmission unit 110a constituting the server 100a receives the SMIL data request message Mdr transmitted from the client terminal 200a by HTTP, reads the SMIL file Da from the data storage unit 120 according to the request, and reads the read SMIL file Da. An HTTP transmission / reception unit 101 that transmits HTTP as SMIL data Dsm, and an RTSP message transmission / reception unit 102 that receives a data request message Mrtsp transmitted from the client terminal 200a by RTSP and outputs a data designation signal Sc indicating the requested video file name. The data designation signal Sc is received, the video stream De corresponding to the video data file name indicated by the data designation signal Sc is read from the data storage unit 120, and the read video stream is And a RTP data transmission unit 103 for transmitting the RTP data Drtp by TP.

  In addition, the client terminal 200a transmits a user operation unit 213 that outputs various user operation signals Sop1, Sop2, and Serr according to a user operation, and transmits a request message Mdr of the SMIL data based on the user operation signal Sop1 by HTTP. In addition, the HTTP transmission / reception unit 211 that receives the SMIL data Dsm transmitted from the server 100a by HTTP, and the SMIL data Dsm are analyzed, and the analysis result and the error tolerance strength set by the user operation are specified. And a SMIL data analysis unit 212 for outputting a data designation signal Sc for designating predetermined data based on a level signal Serr indicating a typical level (numerical value).

  Here, based on the level signal Serr, the SMIL data analysis unit 212 determines necessary data out of a plurality of video data prepared on the server side and having different I-frame periods. A designation signal Sc for designating video data is output.

  The client terminal 200a transmits the data designation signal Sc as an RTSP message signal Mrtsp, receives the RTSP message transmission / reception unit 214 that receives the response signal Sack of the signal Mrtsp, and the RTP data Drtp transmitted from the server 100a. The RTP data receiving unit 216 that outputs the video stream De, the decoding unit 210 that decodes the video stream De and outputs the image data Ddec, displays an image based on the image data Ddec, and performs the above user operation And a display unit 218 that performs display in accordance with the signal Sop2.

Hereinafter, the configuration for setting the error tolerance in the user operation unit 213 will be specifically described.
FIG. 4A shows a screen (error tolerance setting screen) for setting the error tolerance strength of image data to be acquired in the receiving terminal 200a. Here, the receiving terminal 200a is assumed to be a mobile terminal 201a such as a mobile phone.
For example, by operating the button operation unit 21 of the mobile terminal 201a, an item [setting] for performing various initial settings is selected from among a plurality of items in the initial menu of the terminal, and more specific items [ When the streaming reception setting] and the item [error tolerance strength setting] are sequentially selected, an error tolerance setting screen 22b shown in FIG. 4A is displayed at the center of the display panel 22 of the mobile phone.

In FIG. 4A, 22a is a screen showing the radio wave intensity, 22c is a screen for guiding the operation, and the screen 22c is operated by operating the up and down cursor keys 21a and 21c of the button operation unit 21. It is shown that the level of error resilience shown on the error resilience setting screen 22b is selected and the selected level should be confirmed by operating the confirm button 21e.
The error tolerance setting screen 22b sets either a preset error tolerance strength [high level] or a preset error tolerance strength [low level] as the error tolerance strength level of the image data to be acquired. It is a screen to do. Further, in the mobile terminal 201a, error tolerance strength [high level] and [low level] are associated with 80 and 20 out of integer values from 0 to 100 as error tolerance strength values, respectively. Then, either the error tolerance strength [high level] or the error tolerance strength [low level] is selected by the user operation, that is, the up / down cursor keys 21a, 21c of the button operation unit 21, and the confirmation button 21e is operated, When the selected level is confirmed, the error tolerance strength value corresponding to the established level is held as the error tolerance strength value of the terminal.

Next, the operation will be described.
In this data transmission system 10a, as shown in FIG. 1B, a SMIL request signal Sd1 (SMIL request message Mrd shown in FIG. 3) for requesting SMIL data is transmitted from the receiving terminal 200a to the server 100a by HTTP. As a response, the SMIL data Dsm is transmitted from the server 100a to the receiving terminal 200a by the HTTP signal Dsd.
Thereafter, the receiving terminal 200a performs processing for transmitting a message Mrtsp specifying a required video stream to the server 100a as the RTSP signal Sd2 based on the analysis result of the SMIL data Dsm and the contents of the user setting. Then, after the response signal Sack is transmitted from the server 100a to the receiving terminal 200a by RTSP, a predetermined video stream Dstr is transmitted from the server 100a to the receiving terminal 200a as RTP data Drtp.

Hereinafter, the data transmission process between the server 100a and the receiving terminal 200a will be described in detail.
First, in the receiving terminal (client terminal) 200a, various settings are performed by a user operation on the user operation unit 213 before requesting SMIL data corresponding to desired image data.
For example, when the receiving terminal 200a is the portable terminal 201a shown in FIG. 4A, the user operates the button operation unit 21 of the portable terminal 201a to select various types of items in the initial menu of the terminal. An item [setting] for performing the initial setting is selected, and further more specific items [streaming reception setting] and item [error tolerance strength setting] are sequentially selected. Then, according to the operation signal Sop2, an error tolerance setting screen 22b shown in FIG. 4A is displayed on the display unit 218, that is, the display panel 22 of the portable terminal.

On this error tolerance setting screen 22b, error tolerance strength [high level] and error tolerance strength [low level] are displayed as candidates for level selection of error tolerance strength of image data to be acquired.
For example, the error tolerance strength [low level] is selected by the user operating the up / down cursor keys 21a and 21c of the button operation unit 21, and the selected error tolerance strength [low level] is confirmed by operating the confirm button 21e. Then, an integer value “20” corresponding to the error tolerance strength [low level] is held as the error tolerance strength value of the mobile terminal.

  Then, when the user displays an image data selection screen (not shown) on the display unit 218 of the receiving terminal 200a and performs an operation of specifying the image data to be acquired on the image data selection screen, The operation signal Sop1 is input to the HTTP transmission / reception unit 211, and the HTTP transmission / reception unit 211 requests a signal Sd1 (SMIL request message Mdr shown in FIG. 3) for requesting SMIL data related to the designated image data (FIG. 1B). Is transmitted to the server 100a. Then, in the server 100a, the HTTP transmission / reception unit 101 receives the SMIL data request signal Sd1 from the client terminal 200a, and the HTTP transmission / reception unit 101 receives the SMIL data request signal Sd1 from the data storage unit 120. A process of reading the SMIL file Da and transmitting it as SMIL data Dsm by HTTP is performed. The SMIL data Dsm is transmitted to the receiving terminal (client terminal) 200a via the network 11, and is received by the HTTP transmitting / receiving unit 211.

  Then, in the receiving terminal 200a, the received SMIL data Dsm is analyzed by the SMIL data analysis unit 212, and one of the four video data files that best matches the contents set by the user is selected, and the selected video is selected. A designation signal Sc indicating a data file is output to the RTSP message transmission / reception unit 214. In the RTSP message transmission / reception unit 214, processing for transmitting the designation signal Sc to the server 100a as RTSP message signal Mrtsp by RTSP is performed.

Hereinafter, the process of selecting the video data file corresponding to the error resilience level set by the user from the four video data files described in the SMIL file by the SMIL data analysis unit 212 will be specifically described. .
First, the SMIL data analysis unit 212 performs a process of digitizing each video element 701 to 704 in the SMIL file.
Specifically, when N (N: natural number) video elements are described in the SMIL file, the numerical value level Y (Y: Y: Y) is calculated for each video element based on the following calculation formula (1). An integer of 0 or more).
Y = 100 · (n−1) / (N−1) (1)
Here, the digitization level Y is a value given to the nth video element from the one with the lower error resistance strength of the corresponding video data file among the N video elements.
If the calculated value calculated by the above calculation formula (1) is not an integer value, the numerical value level Y is set to an integer value that is equal to or larger than the calculated value and is closest thereto.
Here, since N = 4, the integer values “100”, “67”, “33”, and “0” are assigned to the four video elements 701 to 704 in order from the one with the highest error resistance strength. That is, the video element 704 has an integer value Yv4 (= 100), the video element 703 has an integer value Yv3 (= 67), the video element 702 has an integer value Yv2 (= 33), and the video element 701 Is assigned an integer value Yv1 (= 0).

  When N = 2, an integer value “100” is assigned to the corresponding video element with higher error tolerance strength, and an integer value “0” is assigned to the corresponding video element with lower error tolerance strength. Is done. When N = 3, the integer values “100”, “50”, and “0” are assigned to the three video elements in order from the higher error tolerance strength, and when N = 5, Integer values “100”, “75”, “50”, “25”, and “0” are assigned to the five video elements in order from the corresponding higher error tolerance strength.

  Then, the error tolerance strength value (user setting value) Xus1 (= 20) of the image data to be acquired, which is set by the user on the mobile terminal, and the adjustments given to each of the video elements 701 to 704 described above. The video element 702 to which the integer value Yv2 (= 33) closest to the user set value Xus1 (= 20) of the error resilience strength is given is selected (FIG. 2B). )reference).

  As described above, the receiving terminal 200a designates a video data file with different error tolerance indicated in the SMIL file according to the user setting at the receiving terminal, and indicates the designated video data file. When the designation signal Sc is transmitted to the server 100a as the RTSP message signal Mrtsp, in the server 100a, the RTSP message signal Mrtsp from the receiving terminal 200a is received by the RTSP message transmission / reception unit 102, and the designation signal Sc is received by the RTP data transmission unit. 103. Then, the transmission unit 103 performs a process of selecting a predetermined video file from a plurality of video files stored in the data storage unit 120 based on the designation signal Sc and transmitting it as RTP data Drtp. .

  When the RTP data Drtp is transmitted to the receiving terminal 200a via the network 11, the RTP data Drtp is received by the RTP data receiving unit 216, and the video stream De is output to the decoding unit 210. Is done. The decoding unit 210 generates image data Ddec by decoding the video stream De, and outputs it to the display unit 218. The display unit 218 performs image display based on the image data Ddec.

  As described above, in the data transmission system 10a according to the first embodiment, the server 100a stores a plurality of video streams having different I-frame periods as encoded data of image data corresponding to the same image series. 120 and a data transmission unit 110 that transmits a predetermined video stream of the plurality of video streams in response to a designation signal Sc from the reception terminal, and the reception terminal 200a is set based on the setting contents of the user. Thus, the designation signal Sc for designating one having a required error tolerance among the plurality of video streams prepared on the server 100a side is transmitted to the server 100a. Make the video stream provided by the site more resistant to transmission errors or video It is possible to choose a good quality.

In the first embodiment, <video> is used as a description element indicating a description regarding each video file in the SMIL data. However, this may be <ref>.
In the first embodiment, RTSP is used as a protocol for requesting data, and RTP is used as a protocol for transmitting video data. However, other protocols may be used.
Further, in the first embodiment, the case has been described in which information relating to a plurality of video streams with different encoding conditions prepared in the server is included in SMIL data, but the information relating to the plurality of video streams is SDP. (Session description protocol) data, MPEG-4 System data (Scene description data in MPEG-4), etc. may be transmitted.

Furthermore, in the first embodiment, the case where the error resilience strength of the video stream is indicated by the I frame period has been described, but the error resilience strength of the video stream is defined by the MPEG-4 video coding standard other than the I frame period. It may be indicated by information for describing various error resilience modes.
For example, the information for describing the error resilience mode of the video stream includes information indicating the size of the video packet in the video stream, or whether or not HEC (Head Extension Code) is used (that is, VOP header information is included in the video packet header). Information indicating whether or not data partitioning (that is, placing important information at the head of the packet) or RVLC (Reversible Variable Length Code), that is, the head of the packet. It may be information indicating whether or not a data structure capable of decoding a variable-length code is used not only from the rear end.

  In the first embodiment, as an attribute included in each video element item, an i-frame-interval attribute or a system-error-resilient-level (error-protection-1eve1) that directly indicates error resilience strength. ) Attributes are shown, but these attribute values may be converted in advance to integer values of 0 to 100 proportional to the level of error resilience strength. In this case, as in Embodiment 1 above In addition, it is not necessary to digitize the attribute value related to the error resilience strength with an integer value of 0 to 100 at the receiving terminal.

  Further, in the first embodiment, as a method for setting the level of error tolerance strength of image data to be received, a method of selecting either error tolerance strength [high level] or error tolerance strength [low level] (FIG. 4 (a)), the method of setting the level of error resilience strength of image data to be received at the receiving terminal is a method of specifying the level of error resilience strength within a certain range using a slide bar or the like. It may be.

FIG. 4B is a diagram for explaining the portable terminal 201b that sets the level of error tolerance strength using a slide bar, and shows an error tolerance setting screen 22d in the portable terminal 201b. In FIG. 4B, the same reference numerals as those in FIG. 4A indicate the same reference numerals as those in the portable terminal 201a of the first embodiment.
For example, by operating the button operation unit 21 of the mobile terminal 201b, as with the operation on the mobile terminal 201a in the first embodiment, various initial settings among a plurality of items in the initial menu of the terminal are performed. When the item [setting] is selected, and further more specific items [streaming reception setting] and item [error tolerance strength setting] are sequentially selected, an error tolerance setting screen 22d shown in FIG. Displayed in the center of the display panel 22 of the terminal, a screen 22e for guiding operation is displayed below the error tolerance setting screen 22d.

  Here, the error tolerance setting screen 22d is a screen for setting the level of error tolerance strength of the image data to be acquired by the slide bar 22d1. The error tolerance setting screen 22d shows a range in which the slide bar 22d1 can be moved in the left-right direction, and the left end position Lp and the right end position Rp in the movement range 22d2 are error tolerance strengths [minimum level]. , A position for designating error tolerance strength [highest level], and an intermediate point Mp between the left end position Lp and the right end position Rp is a position for designating error tolerance strength [medium level].

And in the user operation part 213 of this portable terminal 201b, the integer value of 0-100 is calculated as an error tolerance strength level based on the following formula (2) according to the position of a slide bar.
X = Ls · (1 / Rs) · 100 (2)
Here, X is an error resistance strength level, Rs is a distance (slide length) between the left end position Lp and the right end position Rp in the slide range 22d2, and Ls is a distance (slide) of the slide bar 22d1 from the left end position Lp. Distance).
For example, when the slide length Rs is 50 mm and the slide distance Ls of the slide bar 22d1 is 15 mm, the error tolerance strength level X is Xus1 (= (15/50) · 100 = 30 from the calculation formula (2). ) If the calculated value of the error resilience level calculated from the calculation formula (2) is not an integer value, the error resilience strength level is set to an integer value that is equal to or greater than the calculated value.

Further, on the screen 22e, by operating the left and right cursor keys 21b and 21d of the button operation unit 21, the slide bar 22d1 shown on the error resistance setting screen 22e is moved to specify the level of error resistance strength, and It is indicated that the level of the specified error resilience strength should be confirmed by operating the confirm button 21e of the button operation unit 21.
When the slide distance Ls of the slide bar 22d1 is designated by the user operation, that is, the left and right cursor keys 21b and 21d of the button operation unit 21, and the designated slide distance is confirmed by the operation of the confirm button 21e, The error tolerance strength is calculated based on the above formula (2), and the calculated value is held as the error tolerance strength value of the mobile terminal.

  Also in this case, the video elements 711 to 714 are based on the error tolerance strength value (user setting value) Xus1 (= 30) of the image data to be acquired, which is set by the user on the mobile terminal. In the process of determining one of them, as shown in the first embodiment, the video element 712 to which the integer value Yv2 (= 33) closest to the user setting value Xus1 of the error tolerance strength is assigned is selected. (See FIG. 2 (b)).

  Note that the process of determining one of the video elements 711 to 714 based on the user setting value is given an integer value closest to the user setting value Xus1 as in the first embodiment. Not only the process in which the video element is selected, but as shown in FIG. 5B, for example, an integer value Yv3 (= 67) that is equal to or greater than the user setting value Xus2 (= 40) and the setting value is assigned. The selected video element 713 may be selected.

  In the first embodiment, the case where the user sets the error tolerance strength for the image data to be received at the receiving terminal has been described. However, the receiving terminal sets the error tolerance strength for the image data to be received, It may be automatically set according to the state of the received radio wave.

  Further, in the first embodiment, the plurality of pieces of image data having different error tolerance corresponding to the same image series are shown having different appearance intervals of the encoded data corresponding to the I frame. The plurality of different image data may have different frame rates, may have different transmission protocols for the image data, or may have different data unit sizes when packetized.

  For example, image data with a high frame rate has higher error tolerance strength than image data with a low frame rate, and image data transmitted by a transmission protocol including retransmission and duplicate transmission includes retransmission and duplicate transmission. Compared to image data transmitted with no transmission protocol, the error tolerance strength is high. Also, image data with a small data unit at the time of packetization has higher error resistance strength than image data with a large data unit at the time of packetization.

Hereinafter, a plurality of image data having different data unit sizes when packetized will be described in detail.
FIG. 6 shows first and second images having different data unit sizes when packetized, which is obtained by encoding a digital video signal Sdv as two image data having different error tolerance corresponding to the same image series. Encoded data is shown.

  That is, the first image encoded data D1 shown in FIG. 6A is a digital video signal corresponding to each of the frames F1 to F3. The encoder Enc converts one frame of encoded data into one video packet. The error tolerance strength obtained by encoding to be stored in VPa1 is low. In the first encoded image data D1 having low error tolerance, when a transmission error occurs during transmission of encoded data corresponding to the frame F2, encoded data of the packet VPa1 including the error part Perr, that is, All the encoded data of the frame F2 cannot be decoded.

Also, the second image encoded data D2 shown in FIG. 6B is a digital video signal corresponding to each of the frames F1 to F3, and the encoded data corresponding to one frame is 3 by the encoder Enc. It is one with high error tolerance obtained by encoding so as to be distributed and stored in one video packet VPb1 to VPb3. In such second encoded image data D2 having high error tolerance, even if a transmission error occurs during transmission of encoded data corresponding to the frame F2, the encoding corresponding to the packet VPb3 including the error part Perr is performed. The encoded data corresponding to the other packets VPb1 and VPb2 can be decoded only by being unable to decode the data.
Note that the encoded image data is not limited to data that is packeted for each frame or data unit that is smaller than the frame as described above, but is data that is packetized for each data unit that is larger than the frame. Also good.

(Embodiment 2)
FIG. 7 is a diagram for explaining the data transmission system according to the second embodiment of the present invention, and shows the configuration of the server and client terminal of the system.
The data transmission system 10b according to the second embodiment replaces the client terminal 200a in the system 10a according to the first embodiment with the error tolerance strength of image data to be received and the RTP data from the server 100a set by the user. A client terminal 200b that determines a video stream having an optimal error resilience strength based on the rate of occurrence of a transmission error of Drtp and transmits a designation signal Sc that designates the determined video stream to the server 100a is provided. .

  That is, the receiving terminal 200b according to the second embodiment assumes that the image data received first is selected from a plurality of video data files indicated in the SMIL file based on the error tolerance strength set by the user. After the start, according to the error occurrence rate of the received image data, the image data having a predetermined error resilience strength being received is switched to one selected from a plurality of video data files indicated in the SMIL file. .

Hereinafter, the client terminal 200b of the second embodiment will be described in detail.
The client terminal 200b includes an RTP data receiving unit 216b and an SMIL data analyzing unit 212b that perform different operations from the RTP data receiving unit 216 and the SMIL data analyzing unit 212 in the client terminal 200a. The HTTP transmission / reception unit 211, the RTSP message transmission / reception unit 214, the decoding unit 210, the user operation unit 213, and the display unit 218 in the client terminal 200b are the same as those in the client terminal 200a of the first embodiment.

  The RTP data receiving unit 216b receives the RTP data Drtp, outputs the time stamp information Its of the RTP packet in the RTP data Drtp, further detects the rate of occurrence of transmission errors of the RTP data, and this error rate Is output. Also, the SMIL data analysis unit 212b converts the video stream supplied from the server as RTP data according to the comparison result between the error occurrence rate indicated by the error signal Rerr and a certain threshold value, in accordance with the encoding condition (that is, error tolerance). A designation signal Sc for switching to another video stream having a different intensity is output to the RTSP message transmission / reception unit 214. The certain threshold is a terminal-specific reference value set in advance for the receiving terminal 200b.

  Here, the RTP data receiving unit 216b calculates the packet loss rate as an error occurrence rate based on the sequence number information included in the header portion of the RTP packet (RTP data). Also, the SMIL data analysis unit 212b selects a video stream with a short I frame period when the packet loss rate increases, and selects a bit stream with a long I frame period when the packet loss rate is low. A designation signal Sc for output is output.

Hereinafter, the calculation of the error occurrence rate will be specifically described.
The RTP packets are given consecutive sequence numbers in the packet transmission order indicated by the sequence number information included in the header portion. The RTP receiving unit 216b displays the total number Na of RTP packets to be received every certain unit time, the sequence number of the RTP packet received at the beginning of the unit time, and the sequence number of the RTP packet received at the end of the unit time. And the total number Nr of RTP packets actually received within this unit time is counted, and the error occurrence rate Erate at that time is obtained by the following calculation formula (3).
Erate = Nr / Na (3)
Next, the operation will be described.
The operation of the data transmission system 10b of the second embodiment is different from the operation of the data transmission system 10a of the first embodiment only in the operations of the SMIL data analysis unit 212b and the RTP data reception unit 216b of the receiving terminal 200b.
That is, in the receiving terminal 200b, as in the receiving terminal 200a of the first embodiment, various settings are performed by a user operation on the user operation unit 213 before requesting SMIL data corresponding to desired image data.
That is, the user sets the level of the error tolerance strength of the image data to be received on the error tolerance setting screen 22b shown in FIG. When the user performs an operation of designating image data to be acquired on an image data selection screen (not shown), an operation signal Sop1 corresponding to this operation is input to the HTTP transmission / reception unit 211, and the HTTP transmission / reception unit 211 Is transmitted to the server 100a a signal Sd1 (SMIL request message Mdr) (see FIG. 1B) requesting SMIL data related to the designated image data.

  Then, in the server 100a, the HTTP transmission / reception unit 101 receives the SMIL data request signal Sd1 from the receiving terminal 200b, and the HTTP transmission / reception unit 101 stores the SMIL file Da corresponding to the SMIL data request signal Sd1 in the data A process of reading from the storage unit 120 and transmitting it as SMIL data Dsm by HTTP is performed. The SMIL data Dsm is transmitted to the receiving terminal 200b via the network 11, and is received by the HTTP transmitting / receiving unit 211.

  In the receiving terminal 200b, the received SMIL data Dsm is analyzed by the SMIL data analyzing unit 212b, and the best video data file selected from the four video data files is selected, and the selected video data file is selected. Is output to the RTSP message transmission / reception unit 214. The RTSP message transmission / reception unit 214 performs processing for transmitting the designation signal Sc to the server 100a as the RTSP message signal Mrtsp by RTSP.

  Then, in the server 100 a, the RTSP message signal Mrtsp from the receiving terminal 200 b is received by the RTSP message transmission / reception unit 102, and the designation signal Sc is output to the RTP data transmission unit 103. Then, the transmission unit 103 performs a process of selecting a predetermined video file from a plurality of video files stored in the data storage unit 120 based on the designation signal Sc and transmitting it as RTP data Drtp. .

  When the RTP data Drtp is transmitted to the receiving terminal 200b via the network 11, the RTP data Drtp is received by the RTP data receiving unit 216b and the video stream De is output to the decoding unit 210. Is done. The decoding unit 210 generates image data Ddec by decoding the video stream De, and outputs it to the display unit 218. The display unit 218 performs image display based on the image data Ddec.

  As described above, in the state where the RTP data Drtp is transmitted from the server 100a to the receiving terminal 200b, the RTP data receiving unit 216b detects the occurrence rate of the transmission error of the RTP data Drtp, and an error indicating the error occurrence rate The signal Rerr is output to the SMIL data analysis unit 212b.

  Then, in the SMIL data analysis unit 212b, the video supplied from the server 100a as RTP data based on the comparison result between the error occurrence rate indicated by the error signal Rerr and a certain threshold value that is a reference value unique to the receiving terminal 200b. A designation signal Sc for switching the stream to other video data having different encoding conditions (that is, error resilience strength) is output to the RTSP message transmission / reception unit 214. Then, the RTSP message transmission / reception unit 214 performs processing for transmitting the designation signal Sc to the server 100a as the RTSP message signal Mrtsp by RTSP.

  In the server 100 a, the RTSP message signal Mrtsp from the receiving terminal 200 b is received by the RTSP message transmission / reception unit 102, and the designation signal Sc is output to the RTP data transmission unit 103. Then, the transmission unit 103 performs processing for selecting a video file indicated by the designation signal Sc from a plurality of video files stored in the data storage unit 120 and transmitting it as RTP data Drtp.

Hereinafter, a process for calculating the error occurrence rate during the transmission of the image data and a process for switching the stream in accordance with the calculated error occurrence rate will be described in detail.
The SMIL data analysis unit 212b is a work memory (not shown) that records information about each video element described in the SMIL file and information indicating the reception state of image data (video stream) corresponding to each video element. )have.

FIG. 8A shows information recorded in the work memory.
Here, in the work memory, information regarding the video elements 711 to 714 in the SMIL file FSD2 shown in FIG. 5A is recorded, and the number of items (number of entries) recorded in the memory is as follows. This corresponds to the number of elements described between the <switch> element 731a and the </ switch> element 731b in the SMIL file FSD2 (that is, the number of video elements).

As shown in FIG. 8A, each item (entry) has a URL (server address) indicating the location of the corresponding video stream on the network, an error resilience strength of the corresponding video stream, An execution flag indicating whether the video stream to be received is in a reception (playback) state that is received and played back, or a non-reception (non-playback) state in which neither reception nor playback is performed, and a corresponding video stream, Includes the latest timestamp.
In the item E2 of the entry number [2], the value of the execution flag is “1”, which indicates that the video stream corresponding to the item E2 is currently being received (reproduced). ing. Further, in the items E1, E3, E4 of the entry numbers [1], [3], [4], the value of the execution flag is “0”, which corresponds to these items E1, E3, E4. This indicates that the video stream to be received (reproduced) is not currently being performed.

  In addition, the error tolerance strength values in the items E1 to E4 are “0”, “33”, “67”, and “100”, and these values are as described in the first embodiment. This is calculated based on the value of the system-error-resilient-level attribute in the SMIL file FSD2 using the calculation formula (1).

In addition, the latest time stamp in each item E1 to E4 is updated as needed by the time stamp given to the header of the latest received RTP packet, and the video stream corresponding to the specific item is changed to the other item. When switching to a video stream corresponding to, the data request timing is used.
In FIG. 8A, the value of the latest time stamp in the items E1, E3, E4 is “0”, and this value “0” indicates that the video stream corresponding to these items has not been received yet. ing. In addition, the latest time stamp value in the item E2 is “3060000”. In MPEG-4, since the time stamp is set using a clock of 90 kHz, this value “3060000” corresponds to 34 seconds.

FIG. 8B shows an association between the error occurrence rate and the error tolerance strength in the receiving terminal 200b.
Information relating to this association is recorded as table information Rte unique to the receiving terminal in the information storage unit (not shown) of the SMIL data analysis unit 212b. Here, the error occurrence rate (threshold) Eth (Eth = 0) percent, Eth (0 <Eth ≦ 3) percent, Eth (3 <Eth ≦ 6) percent, and Eth (6 <Eth) percent are the error tolerance strengths, respectively. The video stream with the lowest error tolerance, the video stream with the error tolerance quantification level “30”, the video stream with the error tolerance quantification level “60”, and the video stream with the highest error tolerance strength is doing. That is, in this table information, the error occurrence rates of 0%, 3%, and 6% are threshold values for switching video streams according to the error occurrence rate.

Next, the operation of the SMIL data analysis unit 212b when switching video streams according to fluctuations in the error rate will be described.
The setting value Xus2 of the error resilience strength at the receiving terminal is “40” as shown in FIG. 5B, and among the video streams corresponding to each video element shown in the SMIL file FSD2. Suppose that the error resilience strength digitization level closest to the error resilience strength setting value Xus2 is selected as the video stream to be received. The numerical level Y of error resilience strength given to each video element shown in the SMIL file FSD2 is calculated by the above formula (1). That is, the video element 714 has an integer value Ys4 (= 100), the video element 713 has an integer value Ys3 (= 67), the video element 712 has an integer value Ys2 (= 33), and the video element 711 has an integer value. Numerical value Ys1 (= 0) is given. Accordingly, the receiving terminal 200b requests and receives a video stream corresponding to the video element 712 and having a numerical value level Y of error resilience Ys2 (= 33) as the first received video stream. .

First, the SMIL data analysis unit 212b of the receiving terminal 200b writes the execution flag value “1” corresponding to the entry [2] into the work memory.
Then, the RTSP message transmission / reception unit 214 of the receiving terminal 200b performs processing for transmitting the data request message for requesting the video stream corresponding to the entry [2], that is, the video stream indicated by the video element 712, by RTSP.

Thereafter, when a video stream corresponding to the video element 712 is input to the receiving terminal 200b, the RTP data receiving unit 216b receives the video stream corresponding to the video element 712, and first receives the video stream corresponding to the video stream. The time stamp information Its of the RTP packet is output to the SMIL data analysis unit 212b.
Then, in the SMIL data analysis unit 212b, the time stamp value corresponding to the entry [2] recorded in the work memory is sequentially updated to the latest value.

  If the error occurrence rate is zero as a result of observing the reception status at the RTP data reception unit 216b for a certain time (for example, 10 seconds), the SMIL data analysis unit 212b displays the table information shown in FIG. 8B. Based on Rte, a video stream having the lowest error resilience strength is selected from the video streams indicated in the SMIL file, and a designation signal designating this video stream as image data to be received is transmitted and received by the RTSP message. Is output to the unit 214.

At this time, the SMIL data analysis unit 212b performs a process of changing the value of the execution flag corresponding to the entry [2] to “0” and the value of the execution flag corresponding to the entry [1] to “1”. .
Thereafter, the RTSP message transmission / reception unit 214 makes a data request to the URL (server address) corresponding to the entry [1] by RTSP, and at this time, based on the latest time stamp corresponding to the entry [2]. The head position of the requested data (video stream) is designated.

FIG. 9 is a diagram showing an example of a sequence by RTSP, that is, message exchange.
When switching video streams, first, the RTSP message transmission / reception unit 214 of the receiving terminal 200b receives a DESCRIBE request for the video stream indicated by the video element 711 by RTSP for the URL (server address) corresponding to the entry [1]. A message (DESCRIBE rtsp: //s.com/s1.mp4 RTSP / 1.0) Sm1 is transmitted. Then, a response message (RTSP / 1.0 200 OK) Rm1 to the DESCRIBE request message Sm1 is transmitted from the RTSP message transmitting / receiving unit 102 of the server 100a corresponding to the URL to the receiving terminal 200b. This response message Rm1 includes SDP data Dsd for the video stream indicated by the video element 711.

  Subsequently, from the RTSP message transmission / reception unit 214 of the receiving terminal 200b, the first SETUP request message (SETUP rtsp) for the video stream indicated by the video element 711 is received by RTSP for the URL (server address) corresponding to the entry [1]. : //s.com/s1.mp4/trackID=1 RTSP / 1.0) Sm2 and second SETUP request message (SETUP rtsp: //s.com/s1.mp4/trackID=2 RTSP / 1.0) Sm3 sent Is done. Then, the RTSP message transmitting / receiving unit 102 of the server 100a corresponding to the URL sends response messages (RTSP / 1.0 200 OK) Rm2, Rm3 to the first and second SETUP request messages Sm2, Sm3 to the receiving terminal 200b. Sent.

  Thereafter, from the RTSP message transmission / reception unit 214 of the receiving terminal 200b, a PLAY request message (PLAY rtsp: // s) for the video stream indicated by the video element 711 is received by RTSP for the URL (server address) corresponding to the entry [1]. .com / s1.mp4 RTSP / 1.0) Sm4 is transmitted. In the case of a PLAY request, the head position of the request data is designated by information (Range: npt = 37-). Since the time stamp value of the latest received RTP packet for the currently received video stream indicates that the display time for the video stream is 34 seconds, the start position of the request data is 34 seconds or later. Here, assuming that the processing delay time for switching the video stream is about 3 seconds, the head position of the request data is the position where the display time is 37 seconds.

  In response to the PLAY request message Sm4, a response message (RTSP / 1.0 200 OK) Rm4 is transmitted from the RTSP message transmitting / receiving unit 102 of the server 100a corresponding to the URL to the receiving terminal 200b. At the same time, the RTP transmitting unit 103 of the server 100a starts processing to transmit the RTP packet of the video stream (video element 711) to the receiving terminal by RTP (time Ts2), and the RTP data receiving unit of the receiving terminal 200a In 216b, processing for receiving the RTP packet is started (time Tr2).

  Also, in the RTSP message transmission / reception unit 214, whether or not the time stamp of the RTP packet for the entry [1] received by the RTP data reception unit 216b is equal to or smaller than the time stamp value of the RTP packet for the entry [2]. If the time stamp of the RTP packet for entry [1] is less than or equal to the time stamp value of the RTP packet for entry [2], a TEARDOWN request message Sm5 is sent to the server for entry [2]. Processing to issue is performed. At the same time, the process of receiving the RTP packet for entry [2] is stopped (time Tr3).

  In other words, the display time (T1) calculated from the time stamp value of the first received RTP packet corresponding to the video stream (s1.mp4) is already received corresponding to the video stream (s2.mp4). The RTP data receiving unit 216b stops receiving the RTP packet corresponding to the video stream (s1.mp4) only when it is smaller than the display time (T2) calculated from the time stamp value of the latest RTP packet. As a result, when the video stream is switched, the video stream after the switching is played back without interruption following the playback of the video stream before the switching.

  On the other hand, in the server 100a for the entry [2], the RTP data transmission unit 103 receives the TEARDOWN request message (TEARDOWN rtsp: //s.com/s2.mp4 RTSP / 1.0) Sm5 and receives the RTP packet for the entry [2]. Is stopped (time Ts3), and a response message Rm5 to the TEARDOWN request message Sm5 is transmitted to the receiving terminal 200b.

The RTP data receiving unit 216b of the receiving terminal 200b discards the RTP packet for the entry [2] having a time stamp that overlaps the time stamp of the RTP packet for the entry [1].
On the other hand, when the error occurrence rate is 5% as a result of observation of the reception status, the error tolerance strength quantification level close to “60” is selected based on the table information Rte shown in FIG. Then, the process of switching the video stream being received to the video stream corresponding to entry [3] is performed.
In FIG. 9, the time Ts1 is the transmission start time of the video stream (s2.mp4), the time Ts4 is the transmission stop time of the video stream (s1.mp4), and the time Tr1 is the video stream (s2.mp4). The reception start time, time Tr4, is the reception stop time of the video stream (s1.mp4).

FIG. 10 is a diagram for explaining video stream switching processing at the receiving terminal, taking a specific RTP packet as an example.
FIG. 10A shows the last received several RTP packets P2 (ks) to P2 (k + 3) stored in the reception buffer corresponding to the video stream (s2.mp4). FIG. 10B shows the first several RTP packets P1 (j) to P1 (j + m) received in the reception buffer corresponding to the video stream (s1.mp4). Here, the display times T2 (k), T2 () calculated from the time stamp values of the RTP packets P2 (k), P2 (k + 1), P2 (k + 2), P2 (k + 3). k + 1), T2 (k + 2), and T2 (k + 3) are 36.00 (seconds), 36.50 (seconds), 37.00 (seconds), and 37.50 (seconds), respectively. Yes, a display time T1 (j calculated from the time stamp values of the RTP packets P1 (j), P1 (j + 1), P1 (j + 2), P1 (j + 3), P1 (j + 4) ), T1 (j + 1), T1 (j + 2), T1 (j + 3), and T1 (j + 4) are 37.00 (seconds), 37.25 (seconds), and 37.50, respectively. (Seconds), 37.75 (seconds), and 38.00 (seconds).

  Specifically, the RTP data receiving unit 216b starts receiving the video stream (s1.mp4) from the RTP packet P1 (j) and receives the video stream (s2.mp4) from the RTP packet P2 (k + 3 ) Ends when it is received. Then, RTP packets P2 (k + 2) and P2 (k + 3) corresponding to the video stream (s2.mp4) whose time stamp value (display time) overlaps that of the video stream (s1.mp4) are discarded. .

FIG. 11 is a diagram showing a flow of video stream switching processing at the receiving terminal.
When the SMIL data analysis unit 212b determines to switch the video stream to be received from the video stream (s2.mp4) to the video stream (s1.mp4) based on the error occurrence rate, the video stream switching illustrated in FIG. Processing begins.

  First, the RTP data receiving unit 216b performs processing for receiving the RTP packet Ps1 corresponding to the switched video stream (s1.mp4), and the SMIL data analyzing unit 212b first receives the variable Ta. The display time calculated from the time stamp value Ts1 of the RTP packet Ps1 (the display time of the data after switching) is set (step S1).

Next, in the SMIL data analysis unit 212b, the display time (data before switching) calculated from the time stamp value Ts2 of the last received RTP packet Ps2 corresponding to the video stream (s2.mp4) before switching is set as the variable Tb. Is set) (step S2).
Next, in the SMIL data analysis unit 212b, the variable Ta, that is, the display time (display time of data after switching) is equal to or less than the variable Tb, that is, the display time (maximum value of display time of data before switching). Whether or not is determined (step S3).

If the result of determination in step S3 is that the variable Ta is not less than or equal to the variable Tb, it is further determined whether or not an RTP packet corresponding to the video stream before switching has been received (step S4).
As a result of the determination in step S4, when the RTP packet corresponding to the video stream before switching is not received, the determination in step S4 is performed again.
On the other hand, as a result of the determination in step S4, when an RTP packet corresponding to the video stream before switching is received, in step S2, the variable Tb is set to the time stamp value Ts2 of the last received RTP packet Ps2. Processing for setting the obtained display time is performed.

  If the variable Ta is equal to or smaller than the variable Tb as a result of the determination in step 3, the RTP data receiving unit 216b receives the RTP packet Ps2 corresponding to the video stream (s2.mp4) before switching. And the processing for discarding the RTP packet Ps2 corresponding to the video stream (s2.mp4) before switching and whose time stamp value overlaps that of the video stream (s1.mp4) is performed, and RTSP In the message transmission / reception unit 214, a request message for stopping transmission of the RTP packet Ps2 corresponding to the video stream (s2.mp4) before switching is issued (step S5).

FIG. 12 is a schematic diagram specifically explaining the processing in the RTSP message transmission / reception unit 214 and the RTP data reception unit 216b of the receiving terminal when the video stream is switched according to the display time.
The error occurrence rate calculation unit 216b1 of the RTP data reception unit 216b performs processing P1 of calculating the error occurrence rate at intervals of, for example, once every 5 seconds during reception of the RTP packet.
Then, for example, when the process P2 for determining switching to the other video stream (for example, s1.mp4) of the currently received video stream (for example, s2.mp4) is performed due to a change in the error occurrence rate (time Tp2) The RTSP message transmission / reception unit 214 performs processing P3 for issuing a DESCRIBE request message, a SETUP request message, and a PLAY request message for the video stream (s1.mp4).

  After that, when the RTP data receiving unit 216b receives the RTP packet P1 (j) for the video stream (s1.mp4), the display time (37.m) corresponding to the time stamp value of the RTP packet P1 (j) received first. 00 seconds) is a display time (37.00 seconds) corresponding to the time stamp value of the latest RTP packet P2 (k + 2) received at this time for the video stream (s2.mp4) before switching. The process P4 to be compared is performed according to the process flow shown in FIG. 11 (time Tp4).

As a result of the comparison process P4, a video stream (s2.mp4) that has a time stamp value corresponding to the video stream (s1.mp4) and that overlaps with the time stamp value of the RTP packet P1 (j) received first. When the RTP packet corresponding to is received, the process P5 for stopping the reception of the RTP packet corresponding to the video stream (s2.mp4) is performed (time Tp5). For this reason, RTP packets P2 (k + 4) to P2 (k + n) transmitted after the reception stop process P5 are not received by this receiving terminal. The display time corresponding to the time stamp value of the received RTP packets P2 (k + 2) and P2 (k + 3) corresponding to the video stream before switching (s2.mp4) is the video stream after switching. Since it is larger than the display time corresponding to the time stamp value of the first received RTP packet P1 (j) corresponding to (s1.mp4), these RTP packets P2 (k + 2) and P2 (k + 3) The RTP data receiving unit 216b discards the data.
Further, in parallel with the reception stop process P5 in the RTP data reception unit 216b, the RTSP message transmission / reception unit 214 performs a process P6 for issuing a TEARDOWN request message for the video stream (s2.mp4).

  In FIG. 12, P2 (kr) is the head RTP packet corresponding to the video stream (s2.mp4), and P2 (k-7) to P2 (k + 3) are the start of the reception stop process P5. Are RTP packets corresponding to the video stream (s2.mp4) received between a few seconds before and immediately before the start of the reception stop process P5. These RTP packets P2 (k-7), P2 (k-6), P2 (k-5), P2 (k-4), P2 (k-3), P2 (k-2), P2 (k-1), P2 (k), P2 (k + 1) Display time 32.50 (second), 33.00 (second), 33.50 (second), 34.00 (second), 34.50 (second), 35.00 (second), 35.50 (second) , 36.00 (seconds), and 36.50 (seconds).

  Further, P1 (j + 1) to P1 (j + 3) are RTP packets corresponding to the video stream (s1.mp4) following the first received RTP packet P1 (j), and these RTP packets P1 Time stamp values corresponding to display times 37.25 (seconds), 37.50 (seconds), and 37.75 (seconds) are assigned to (j + 1) to P1 (j + 3). P1 (j + m) is the last received RTP packet corresponding to the video stream (s1.mp4).

  Note that the timestamp value written in the header of the RTP packet is given its initial value by the timestamp described in the RTP-Info field in the RTSP transmission message. Time stamp values are not simply compared between RTP packets corresponding to streams, but display times corresponding to time stamp values are compared.

The display time Td is calculated by the following calculation formula (4).
Td = Th + (Pts−Ptsi) / Sts (4)
Here, Th is a time indicating the start position of the reproduction data specified in the Range field in the PLAY response message, Pts is a value of a time stamp (packet time stamp) given to each packet, Ptsi is an initial value of the time stamp, Sts is a time scale, and the time scale is specified in the SDP information returned from the server as a response to the DESCRIBE request.

  As described above, in the data transmission system 10b of the second embodiment, the RTP data Drtp from the server 100a is received instead of the RTP data receiving unit 216 of the receiving terminal 200a of the first embodiment, and the received RTP packet An RTP data receiving unit 216b that outputs an error signal Rerr indicating the loss rate (transmission error rate) of the RTP packet at the receiving terminal to the SMIL data analyzing unit 212b by analysis is provided. The data analyzing unit 212b determines the loss rate of the packet. A signal (data designation signal) Sc for instructing the server to switch the video stream provided from the server 100a to one having high resistance to transmission errors or one having high video quality is generated in accordance with the change. Then, when the transmission error rate is high, use it on the server side. Can be received with high error tolerance with a short I-frame period, and when the rate of occurrence of transmission errors is low, the I-frame among the video streams prepared on the server side A video with a long period and high image quality can be received.

  In the second embodiment, the case where the SMIL file is the one (SMIL file FSD2) shown in FIG. 5A that shows four video data files with different error tolerance strengths. As shown in FIG. 13 (a), three video elements having different error resilience strengths are shown. Each video element has an error resilience strength described as a system-protocol attribute (SMIL file FSD3). There may be.

  That is, the SMIL file FSD3 shown in FIG. 13A is an item related to three video elements 721 to 723 having different error resilience strengths described between a line including the switch element 732a and a line including the / switch element 732b. Is included. In addition, in each video element item, error resilience strength is described as a system-protocol attribute. Based on this attribute, the video element that best matches the contents of the user setting is selected.

  Here, the specific values of the system-protocol attribute in each of the video elements 721, 722, and 723 are “nop”, “ret”, and “fec + ret”, respectively. The attribute value “nop” indicates that the video stream (s1.mp4) corresponding to the video element 721 is transmitted by RTP which is a normal data transmission protocol. In addition, the attribute value “ret” causes the video stream (s2.mp4) corresponding to the video element 722 to be retransmitted (ret: retransmission) with error tolerance for RTP, which is a normal data transmission protocol. It is shown that it is transmitted by the method. Further, the attribute value “fec + ret” indicates that the video stream (s3.mp4) corresponding to the video element 723 has higher error resistance than the transmission method for performing retransmission (ret: retransmission) with the above error resistance. This indicates that the transmission is performed by a method of performing retransmission and duplicate transmission (fec = forward error correction).

That is, since the video stream (s1.mp4) corresponding to the video element 721 to which the system-protocol attribute value “nop” is assigned is neither retransmitted nor duplicated, the error tolerance is the above three video. It is the weakest video stream corresponding to the element.
Accordingly, when the error tolerance strength is set to [weak level] at the receiving terminal, a video stream corresponding to the video element 721 is selected as a video stream to be received. If there is no setting of error resilience strength at the receiving terminal, the video stream (s1.mp4) corresponding to the video element 721 is selected as the first received video stream, and the video stream (s1.mp4) When the transmission error rate increases after reception of the video stream, the video stream being received is a video stream corresponding to the video elements 722 and 723 to which the system-protocol attribute value “ret” or “ret + fec” is assigned. It is switched to (s2.mp4) or (s3.mp4).
Note that the video stream (s2.mp4) transmitted by the transmission method that performs retransmission corresponding to the video element 722 is transmitted by the transmission method that performs duplicate transmission, that is, the system-protocol attribute value of the video element May be “fec”.

Further, in the SMIL data analysis unit 212b, when the SMIL file FSD3 shown in FIG. 13A is input, the description information of the SMIL file is changed based on the SMIL file as shown in FIG. 13B. A process of storing in a work memory (not shown) is performed.
That is, information on the video elements 721 to 723 in the SMIL file FSD3 shown in FIG. Here, the number of items (number of entries) recorded in the work memory is the number of elements described between the <switch> element 732a and the </ switch> element 732b in the SMIL file FSD3 (that is, the number of video elements). ).

  In each item (entry), as shown in FIG. 13B, the URL (server address) indicating the location of the corresponding video stream on the network, the transmission protocol of the corresponding video stream, and the corresponding video An execution flag indicating whether the stream is in a received (playback) state that has been received and played back, or a non-receive (non-playback) state that has not been received or played back, and the latest Includes a time stamp.

  In the item E1 of the entry [1], the value of the execution flag is “1”, which indicates that the video stream corresponding to the item E1 is currently being received (reproduced). Yes. In addition, in the items E2 and E3 of the entries [2] and [3], the value of the execution flag is “0”. This is because the video streams corresponding to these items E2 and E3 are currently received ( Replay) is not performed.

Further, specific values indicating the protocol types in the items E1 to E3 are “nop”, “ret”, “fec + ret”, and these values are the system-protocol in the SMIL file FSD3. It matches the attribute value.
In addition, the latest time stamp in each item E1 to E3 is updated at any time by the time stamp given to the header of the latest received RTP packet, and the video stream corresponding to the specific item is changed to another item. When switching to the corresponding video stream, it is used to determine the data request timing.

In FIG. 13B, the value of the latest time stamp in the items E2 and E3 is “0”, and this value “0” indicates that the video stream corresponding to these items has not been received yet. . Further, the value of the latest time stamp in the item E1 is “3060000”. In MPEG-4, since the time stamp is set using a clock of 90 kHz, this value “3060000” corresponds to 34 seconds.
FIG. 13C shows an association between the error occurrence rate and the protocol.

  Information relating to this association is recorded as table information Rtp specific to the receiving terminal in the information storage unit (not shown) of the SMIL data analysis unit 212b. Here, the error occurrence rates Eth (Eth = 0) percent, Eth (0 <Eth ≦ 3) percent, and Eth (3 <Eth) percent are the video stream transmitted by the nop protocol and the video transmitted by the ret protocol, respectively. Supports stream, video stream transmitted by fec + ret protocol. That is, in this table information, the error occurrence rates of 0% and 3% are threshold values for switching the video stream according to the error occurrence rate.

  Then, in the SMIL data analysis unit 212b, switching of the video stream according to the fluctuation of the error occurrence rate is performed based on the association between the error occurrence rate and the protocol illustrated in FIG. Also, switching of video streams for seamless playback is performed in the same manner as the processing described with reference to FIGS.

  In the second embodiment, as the receiving terminal, the user sets the error tolerance strength of the image data to be received first among a plurality of pieces of image data having different error tolerance corresponding to the same image series. Although shown, the error tolerance strength of the image data to be received first may be a default value unique to the receiving terminal.

In this case, for example, the receiving terminal requests the video stream of the video element suitable for the default value of the error resilience strength among the plurality of video elements 711 to 714 indicated by the SMIL file FSD2, and receives the video stream. Thereafter, in the receiving terminal, the video stream being received is switched to a video stream having an appropriate error resilience strength according to the error occurrence rate during the reception of the video stream.
In the second embodiment, the video stream is switched according to the error occurrence rate with respect to the video stream being received. However, the video stream is switched according to the radio wave intensity being received. It may be.

(Embodiment 3)
FIG. 14 is a diagram for explaining the data transmission system according to the third embodiment of the present invention, and shows the configuration of the server and client terminal of the system.
In FIG. 14, the same reference numerals as those in FIG. 3 denote the same components as those in the data transmission system 10a of the first embodiment.

  In the data transmission system 10c of the third embodiment, instead of the client terminal 200a in the system 10a of the first embodiment, a transmission error rate of RTP data (RTP packet) from the server, packet arrival time, and the like are transmitted. A client terminal 200c that transmits information Drr relating to the situation to the server 100c is provided. Further, instead of the server 100a in the system 10a of the first embodiment, RTP data is sent from the server as RTP data based on the information Drr relating to the transmission situation from the client terminal 200c. The server 100c is provided that switches the supplied video stream to another video stream with different encoding conditions.

  The client terminal 200c receives RTP data Drtp instead of the RTP data receiving unit 216a in the client terminal 200a, and detects an RTP data transmission error occurrence rate, an RTP packet arrival time, and other transmission status A data receiving unit 216c is provided, and an RTCP report transmission / reception unit 219 that transmits information Drr indicating the transmission status to the server 100c as a receiver report is provided.

  The server 100c transmits information Dsr related to the number and sequence number of RTP packets transmitted from the server as a sender report to the RTCP report transmission / reception unit 219 of the reception terminal 200c and receives a receiver report from the transmission / reception unit 219. In addition to the report transmission / reception unit 104, it receives information Drr as a receiver report instead of the RTP data transmission unit 103 in the server of the first embodiment, and is based on the transmission status such as the frequency of occurrence of transmission errors and the arrival time of RTP packets. Thus, an RTP data transmission unit 103c that switches a video stream transmitted as RTP data to another video stream with different encoding conditions is provided.

  The RTCP report transmission / reception units 104 and 219 transmit and receive the sender report and the receiver report by RTCP (real time control protocol). In addition, the receiver report is notified to the distribution server at a constant cycle such as every 5 seconds. Moreover, it is preferable that the timing for switching the video stream at the server is generally the timing at which the I frame appears.

Next, the operation will be described.
The operation of the data transmission system 10c according to the third embodiment is based on the receiver report from the receiving terminal 200c, and the video stream transmitted to the receiving terminal as RTP data in the server 100c has different encoding conditions. Only the operation is different from the operation of the data transmission system 10a of the first embodiment.

That is, the RTP data receiving unit 216c of the receiving terminal 200c detects the transmission error occurrence rate of the received RTP data Drtp, and outputs an error signal Rerr indicating the error occurrence rate to the RTCP report transmission / reception unit 219.
From the RTCP report transmission / reception unit 219, information on the frequency of occurrence of transmission errors and the arrival time of the RTP packet is transmitted to the server 100c as a receiver report Drr.

  Then, the RTCP report transmission / reception unit 104 of the server 100c detects the transmission error occurrence rate and the packet arrival delay time of the RTP data Drtp based on the information received as the receiver report Drr. The error occurrence rate and the arrival delay time are detected. Is output to the RTP data transmitter 103c.

  In the RTP data transmission unit 103c, a video file having a predetermined error resistance is selected from a plurality of video files stored in the data storage unit 120 in accordance with the increase / decrease in the error occurrence rate and the packet arrival delay time. , RTP data Drtp is transmitted to the receiving terminal 200c.

  As described above, in the data transmission system 10c according to the third embodiment, instead of the client terminal 200a in the system 10a according to the first embodiment, the rate of occurrence of RTP data (RTP packet) transmission errors from the server, packet arrival time, etc. The client terminal 200c for transmitting information Drr relating to the transmission status of the client to the server 100c, and, instead of the server 100a in the system 10a of the first embodiment, based on the information Drr relating to the transmission status from the client terminal 200c, RTP data Since the server 100c is provided that switches the video stream supplied from the server to another video stream with different encoding conditions, the server 100c has a high transmission error rate based on the receiver report from the receiving terminal 200c. , Double Among a plurality of video streams, one with a high error tolerance with a short I frame period can be transmitted, and when the transmission error occurrence rate is low, a video quality with a long I frame period is selected from among a plurality of video streams. You can send something expensive.

(Embodiment 4)
FIG. 15 is a diagram for explaining the data transmission system of the fourth embodiment, and shows the configuration of the server and client terminal of the system.
In FIG. 15, the same reference numerals as those in FIG. 3 denote the same components as those in the data transmission system 10a of the first embodiment.
The data transmission system 10d of the fourth embodiment includes a client terminal 200d that changes the decoding process and the display process according to the operation content set by the user, instead of the client terminal 200a in the system 10a of the first embodiment. It is a thing.

  That is, the client terminal 200d is a decoding unit 210d that changes an operation mode for performing a video stream decoding process based on the control signal C1, instead of the decoding unit 210 and the display unit 218 of the client terminal 200a of the first embodiment. And a display unit 218d for changing the operation mode for performing the display process of the image data Ddec based on the control signal C2, and the operation modes of the decoder unit 210d and the display unit 218d based on the setting signal Serr indicating the setting contents of the user. The controller 220 is controlled by the control signals C1 and C2.

Next, the operation will be described.
The operation of the data transmission system 10d according to the fourth embodiment is performed only in that the decoding process mode of the video stream and the display process mode of the image data are changed according to the setting contents of the user at the receiving terminal 200d. This is different from the operation of the system 10a of the first embodiment.

  That is, when a video stream to be played back by the receiving terminal 200d is set by a user operation on the user operation unit 213, a decoding unit is set when the I frame cycle is smaller than a certain reference cycle specific to the receiving terminal. 210d is set by the control signal C1 from the control unit 220 to the first decoding operation mode in which the operation mode is temporarily stopped until the I-frame video stream is normally received when a transmission error occurs. The Further, in this case, the display unit 218d causes the transmission signal to be generated until the video stream of the next I frame is normally received by the control signal C2 from the control unit 220 when the operation mode is a transmission error. The first display operation mode for displaying the image data decoded immediately before is set.

  On the other hand, when a video stream to be played back by the receiving terminal 200d is set by a user operation on the user operation unit 213, a decoding unit is set when the I frame cycle is greater than or equal to a certain reference cycle unique to the receiving terminal. 210d is controlled by the control signal C1 from the control unit 220. When a transmission error occurs, 210d skips only the decoding process of the frame in which data is lost due to the transmission error, and after the transmission error occurs, the data is normally transmitted. Is set to the second decoding operation mode in which the decoding process is performed from the received frame. In the second decoding operation mode, when a frame in which data is normally received after the occurrence of a transmission error is a P frame, the decoding process is performed with reference to the frame decoded immediately before the occurrence of the transmission error. Is called. Further, in this case, the display unit 218d displays, based on the control signal C2 from the control unit 220, a second display in which all the frames in which the operation mode is subjected to the data decoding process regardless of the occurrence of the transmission error are displayed. The operation mode is set.

  As described above, in the data transmission system 10d according to the fourth embodiment, the decoding unit 210d and the display unit in the receiving terminal are set according to the conditions regarding the error resistance of the video stream requested by the receiving terminal set by the user at the receiving terminal. If the condition that the operation mode of 218d is changed, that is, the video stream to be received by the receiving terminal is set to a video stream whose I-frame period is shorter than a certain reference value is set, transmission is performed. When an error occurs, the decoding process is temporarily stopped until the I-frame video stream is normally received, the decoded image data is displayed immediately before the occurrence of the transmission error, and the video stream to be received by the receiving terminal Is set to be a video stream whose I frame period is equal to or greater than a certain reference value. In this case, only the decoding process for frames other than the frame in which data is lost due to a transmission error is performed, and all the frames subjected to the data decoding process are displayed. Depending on the error tolerance of the stream (that is, the interval between I frames), the operation mode of the decoding unit and the display unit can be set to an operation mode in which a sense of discomfort of the display image when an error occurs is small.

  In the fourth embodiment, the data transmission system has been described in which the decoding processing mode and the display processing mode in the receiving terminal are changed according to the conditions regarding the video stream set by the user on the receiving terminal side. The data transmission system determines the operation mode of the decoding unit 210d and the display unit 218d at the receiving terminal based on the appearance interval (I frame period) of the I frame related to the video stream transmitted from the server notified from the server. It may be changed. In this case, information indicating the appearance interval of the I frame can be transmitted from the server to the receiving terminal using SMIL, SDP, RTSP, or the like.

  In the fourth embodiment, as the second decoding operation mode of the decoding unit 210d, when a transmission error occurs, only the decoding process of the frame in which data is lost due to the transmission error is skipped, and after the transmission error occurs, Although an operation mode in which a decoding process is performed from a frame in which data is normally received is shown, the second decoding operation mode is not limited to this.

For example, as shown in FIG. 6B, when a video stream of one frame is distributed and stored in a plurality of video packets, the second decoding operation mode, that is, the I frame period is unique to the receiving terminal. The decoding operation mode when a signal having a certain reference period or more is set may be a mode in which only decoding processing is performed on data of packets other than video packets in which data is lost due to a transmission error.
In this case, the display mode of the image data may be a mode in which all frames for which at least a part of the decoding process has been performed are displayed, as in the second display operation mode of the fourth embodiment. .

  Furthermore, in the fourth embodiment, the control unit switches the operation mode of the decoding unit from the first decoding operation mode to the second decoding operation mode according to the user setting at the receiving terminal. However, the control of the operation of the decoding unit by the control unit is not limited to this, and may be performed according to conditions other than user settings at the receiving terminal, for example.

  For example, when a transmission error occurs, the time until the next I-frame video stream is decoded can be calculated because the period of the I-frame is known. Therefore, when a transmission error occurs, the control unit performs the decoding operation of the decoding unit according to the time difference from the time of decoding of the frame in which the transmission error has occurred to the time of decoding of the I frame to be decoded thereafter. For example, between the time of decoding a frame in which a transmission error has occurred and the time of decoding of a subsequent I frame, a decoding operation for stopping the decoding process and a subsequent I frame from the time of decoding of a frame in which a transmission error has occurred Until the time of decoding, it is determined whether to decode the inter-screen encoded data by excluding the part that cannot be decoded due to the occurrence of the transmission error. Control may be performed so that the decoding operation after generation is the decoding operation determined by this determination.

  Specifically, when the transmission error occurs, the time difference between the decoding of the frame in which the transmission error has occurred and the subsequent decoding of the I frame is smaller than a predetermined value unique to the terminal. The decoding operation of the decoding unit stops the decoding process on the image data until the I frame is decoded after the frame in which the transmission error has occurred, while the transmission error has occurred. When the time difference between the decoded frame and the subsequent I frame is equal to or greater than the predetermined value specific to the terminal, the decoding operation of the decoding unit is performed after the decoding of the frame in which the transmission error has occurred. Until the I frame is decoded, only the image data corresponding to the frame other than the frame in which the transmission error has occurred is decoded. , It controls the decode unit.

  Here, when the image data of each frame is packetized for each data unit smaller than the frame as shown in FIG. 6B, the decoding process for the frame other than the frame in which the transmission error has occurred. The decoding operation that performs only the decoding may be performed only on the received image data for packets other than the packet in which the transmission error has occurred.

  Further, in each of the above embodiments, the RTSP is used to notify the server of the viewer's preference regarding the display image (whether a shorter I-frame cycle is better or a longer I-frame cycle is better). Good. In addition, CC / PP (composite capability / preference profiling), which is another transmission protocol, may be used as a protocol for notifying the viewer's preference. At this time, the server may notify the receiving terminal of the compensation of the video stream using SMIL.

  Further, in each of the embodiments described above, the case where the data transmitted from the server to the receiving terminal is video data has been described. However, the transmission data may be audio data or text data. Even when text data is transmitted by RTP / UDP / IP, the same effects as those of the above embodiments can be obtained.

  For example, data to be received corresponding to the same content, set by the user at the receiving terminal or set as the default value of the receiving terminal from a plurality of audio data with different error tolerances or a plurality of text data Is selected according to the error resistance strength against the selected data, and the selected voice data or text data is reproduced at the receiving terminal. Here, as an example in the case where a plurality of audio data (text data) has different error tolerances, one of the plurality of audio data (text data) is an audio frame (text frame) that has been previously decoded. There is a case where a frame that is decoded with reference to data is used and the other one does not use such a frame.

  Further, the plurality of audio data or the plurality of text data corresponding to the same content and having different error resistance strengths may have different data transmission protocols. As an example of a different transmission protocol for voice data or text data, there is one in which the redundancy of FEC (Forward Error Correction, RFC 2733) defined by IETF (Internet Engineering Task Force) is different.

(Embodiment 5)
FIG. 21 is a diagram for explaining a data transmission system according to a fifth embodiment of the present invention. FIG. 21 (a) shows the configuration of the system, and FIG. 21 (b) shows data transmission processing in the system. Is shown.
The data transmission system 10e according to the fifth embodiment includes a server 100e that transmits a predetermined video stream (image encoded data), and a receiving terminal that receives the video stream transmitted from the server 100e and reproduces video data ( Client terminal) 200e and a network 11 for transmitting the video stream from the server 100e to the receiving terminal 200e.

  Here, the server 100e stores a plurality of video streams obtained by encoding digital video signals of a plurality of image sequences under a predetermined encoding condition, and describes attributes of corresponding video streams. The data storage unit 120e stores SMIL data, and the data transmission unit 110e transmits the data stored in the data storage unit 120e to the network 11. The data storage unit 120e uses a mass storage device such as a hard disk.

  In the fifth embodiment, the plurality of video streams are image data corresponding to different image sequences and having determined error tolerances. Specifically, each of the plurality of video streams uses a large amount of intra-screen encoded data obtained by encoding a digital video signal using intra-screen pixel value correlation and a digital video signal using inter-screen pixel value correlation. And the inter-coded data with a small code amount, which is encoded, and has a predetermined appearance interval of the intra-coded data, in other words, a cycle of I frame (I-VOP).

  In the data storage unit 120e such as the hard disk, for example, a video stream having an I frame period of 5 seconds and 2 seconds is stored as video files Dva and Dvb, and the corresponding video is stored as the SMIL data Daa and Dab. SMIL files describing the attributes of the files Dva and Dvb are stored. Here, the appearance intervals of I frames (I-VOPs), which are the attributes of the video streams (video files) Dva and Dvb, are 5 seconds and 2 seconds, respectively.

FIG. 22 is a diagram showing a detailed configuration of the server 100e and the client terminal 200e constituting the system.
The data transmission unit 110e constituting the server 100e receives the SMIL data request message Mdr transmitted from the client terminal 200e by HTTP, reads the SMIL file Da from the data storage unit 120e according to the request, and reads the read SMIL file Da. An HTTP transmission / reception unit 101 that transmits as SMIL data Dsm by HTTP, a data request message Mrtsp transmitted by RTSP from the client terminal 200e, receives a response signal Sack, and a data designation signal indicating the requested video file name The RTSP message transmission / reception unit 102 that outputs Sc and the data designation signal Sc, and receives the video stream De corresponding to the video data file name indicated by the data designation signal Sc from the data storage unit 120e. Read, and a RTP data transmission unit 103 for transmitting the RTP data Drtp by the read video stream RTP. The HTTP transmission / reception unit 101, the RTSP message transmission / reception unit 102, and the RTP data transmission unit 103 in the data transmission unit 110e of the fifth embodiment are the same as those in the data transmission unit 110a of the first embodiment.

  In addition, the client terminal 200e includes a user operation unit 213 that outputs various user operation signals Sop1, Sop2, and Sop3 according to user operations, and SMIL data corresponding to user-specified video data based on the user operation signal Sop1. The HTTP request message Mdr is transmitted by HTTP, the SMIL data Dsm transmitted from the server 100e by HTTP is received, and the SMIL data Dsm is analyzed, and the user designation is made based on the analysis result. A SMIL data analysis unit 212e for outputting a data designation signal Sc for designating the video data.

  The client terminal 200e transmits the data designation signal Sc as an RTSP message signal Mrtsp, receives the RTSP message transmission / reception unit 214 that receives the response signal Sack of the signal Mrtsp, and the RTP data Drtp transmitted from the server 100e. And an RTP data receiving unit 216 that outputs the video stream De.

  Further, the client terminal 200e decodes the video stream De and outputs image data Ddec, and also, based on the control signal C1, a decoding unit 210e that changes an operation mode for performing decoding processing of the video stream, and the image data The display unit 218e that performs image display based on Ddec and changes the operation mode for performing display processing of the image data Ddec based on the control signal C2, and the operation modes of the decoder unit 210e and the display unit 218e are set to the control signal C1 and And a control unit 220e controlled by C2. The display unit 218e also performs display according to the user operation signal Sop2.

  Further, in this client terminal 200e, a default value to be compared with the appearance interval of the encoded image data in the received image data is set as a default value. When an error occurs, the in-screen code in the received image data is set. The operation mode of the decoding unit is switched in accordance with the comparison result between the appearance interval of the encoded data and the predetermined value. Specifically, when receiving image data in which the appearance interval of the intra-picture encoded data is shorter than the predetermined value, the operation mode of the decoding unit is such that when the transmission error occurs, the intra-picture encoded data is thereafter When the first decoding mode in which the decoding process is temporarily stopped until normal reception is received and image data in which the appearance interval of the in-screen encoded data is greater than or equal to a predetermined value indicated by the setting condition is received, the decoding is performed. The operation mode of the conversion unit is a second decoding mode in which, when a transmission error occurs, decoding is performed except for a portion that cannot be decoded due to the transmission error.

  Note that the receiving terminal is not limited to having a default value that is compared with the appearance interval of the intra-frame encoded data in the image data being received as a default value. It may be settable.

Next, the operation will be described.
In this data transmission system 10e, when the user performs an operation for requesting a predetermined video file at the user operation unit 213e, based on the operation signal Sop1, as shown in FIG. A SMIL request signal Sd1 (SMIL request message Mrd shown in FIG. 22) for requesting SMIL data corresponding to a user-specified video file is transmitted from the HTTP transmission / reception unit 211 to the server 100e by HTTP, and as a response, the HTTP of the server 100e The SMIL data Dsm is transmitted from the transmitting / receiving unit 101 to the receiving terminal 200e by the HTTP signal Dsd. Note that the user's operation of designating a video file of a required image sequence on the user operation unit 213e is performed in the same manner as the operation described using the mobile terminal shown in FIG.

  Thereafter, in the receiving terminal 200e, the RTSP message transmission / reception unit 214 uses the server 100e as the RTSP signal Sd2 with the message Mrtsp specifying the video stream required by the user based on the data specifying signal Sc corresponding to the analysis result of the SMIL data Dsm. Process to send to. Then, after the response signal Sack is transmitted from the RTSP message transmitting / receiving unit 102 of the server 100e to the receiving terminal 200e by RTSP, the RTP data transmitting unit 103 receives a predetermined video stream Dstr as RTP data Drtp from the server 100e. It is transmitted to the terminal 200e.

  In this way, when the RTP data Drtp is transmitted to the receiving terminal 200a via the network 11, the RTP data Drtp is received by the RTP data receiving unit 216, and the video stream De is decoded by the decoding unit 200a. It is output to 210e. In the decoding unit 210e, image data Ddec is generated by decoding the video stream De and is output to the display unit 218e. The display unit 218e performs image display based on the image data Ddec.

  In the data transmission system 10e according to the fourth embodiment, when an error occurs during the transmission of the video stream, the reception terminal 200e has an appearance interval of intra-coded data set as a default value ( That is, the operation mode of the decoding unit 210e and the operation mode of the display unit 218e are changed from the control unit 220e according to the comparison result between the period of the I frame and the period of the I frame that is the attribute value of the received video stream. Is changed based on the control signals C1 and C2.

  That is, when the receiving terminal 200e receives a video stream having an I frame period (I-VOP period) shorter than a predetermined value (a constant reference period) at the receiving terminal, the decoding unit 210e performs control. The control signal C1 from the unit 220e sets the operation mode to a first decoding operation mode in which the decoding process is temporarily stopped until a video stream of I frame is normally received when a transmission error occurs. Also, in this case, the display unit 218e causes the transmission error to occur until the video stream of the next I frame is normally received by the control signal C2 from the control unit 220e. The first display operation mode for displaying the image data decoded immediately before is set.

  On the other hand, when the receiving terminal 200e receives a video stream whose I frame period is equal to or greater than a predetermined value (a constant reference period) at the receiving terminal, the decoding unit 210e receives the control signal C1 from the control unit 220e. Therefore, when a transmission error occurs, the operation mode skips only the decoding process of the frame in which data is lost due to the transmission error, and performs the decoding process from the frame in which the data is normally received after the transmission error occurs. The second decoding operation mode is set. In the second decoding operation mode, when a frame in which data is normally received after the occurrence of a transmission error is a P frame, the decoding process is performed with reference to the frame decoded immediately before the occurrence of the transmission error. Is called. Further, in this case, the display unit 218e is a second display that displays all the frames in which the operation mode is the data decoding process regardless of the occurrence of the transmission error by the control signal C2 from the control unit 220e. The operation mode is set.

  As described above, in the data transmission system 10e according to the fifth embodiment, according to the default value of the I frame period set as the default value in the receiving terminal and the value of the I frame period of the received video stream, The operation mode of the decoding unit 210e and the display unit 218e in the receiving terminal is changed, that is, the value of the I frame period of the video stream received by the receiving terminal is greater than the default value set as the default value in the receiving terminal. In a short case, when a transmission error occurs, the decoding process is temporarily stopped until the I-frame video stream is normally received, and the decoded image data is displayed immediately before the transmission error occurs. The I-frame period value of the video stream received at the default is set as the default value at the receiving terminal. If the value is greater than or equal to the value, only the decoding process is performed for frames other than the frame in which data is lost due to a transmission error, and all the frames that have been subjected to the data decoding process are displayed. In accordance with (that is, the interval of the I frame), the operation mode of the decoding unit and the display unit can be reduced in discomfort in the display image when an error occurs.

  In the fifth embodiment, the I frame appearance interval (I frame period), which is the attribute value of the received video stream, is shown as being supplied from the server 100e to the receiving terminal as a SMIL file. The I frame appearance interval (I frame period) of the received video stream may be transmitted from the server to the receiving terminal using SDP, RTSP, or the like.

  In addition, the I-frame appearance interval (I-frame period) of the received video stream is not limited to the case where the I-frame is transmitted from the server to the terminal. For example, the RTP data receiving unit 216 of the receiving terminal 200e You may make it calculate from the information contained.

  In the fifth embodiment, as the second decoding operation mode of the decoding unit 210e, when a transmission error occurs, only the decoding process of the frame in which data is lost due to the transmission error is skipped, and after the transmission error occurs, Although an operation mode in which a decoding process is performed from a frame in which data is normally received is shown, the second decoding operation mode is not limited to this.

For example, as shown in FIG. 6B, when a one-stream video stream is distributed and stored in a plurality of video packets, the second decoding operation mode has lost data due to a transmission error. A mode may be used in which only decoding processing is performed on data of packets other than video packets.
Further, in this case, the display mode of the image data may be a mode in which all the frames that have been subjected to the decoding process on at least a part of the data are displayed, as in the second display operation mode of the fifth embodiment. .

Further, in the fifth embodiment, the operation mode of the decoding unit at the time of error occurrence is changed according to the magnitude relationship between the I-frame appearance interval of the video stream being received and the default value (default value) at the receiving terminal. Although what is switched is shown, the switching of the operation mode of the decoding unit is not limited to this.
For example, when a transmission error occurs, the time until the next I-frame video stream is decoded can be calculated because the period of the I-frame is known. For this reason, when a transmission error occurs, the control unit performs the decoding operation of the decoding unit according to the time difference from the time of decoding of the frame in which the transmission error has occurred to the time of decoding of the I frame to be decoded thereafter. For example, between the time of decoding a frame in which a transmission error has occurred and the time of decoding the subsequent I frame, the decoding operation for stopping the decoding process, Until the time of decoding, it is determined whether to decode the inter-frame encoded data except for the part that cannot be decoded due to the transmission error. The subsequent decoding operation may be controlled to be the decoding operation determined by this determination.

  Specifically, when a transmission error occurs, the control unit determines whether the time difference between the decoding of the frame in which the transmission error has occurred and the subsequent decoding of the I frame is a default value (default) at the receiving terminal. If the value is smaller than (value), the decoding operation of the decoding unit is an operation of stopping the decoding process on the image data from the time of decoding the frame in which the transmission error has occurred until the I frame is decoded thereafter, When the time difference from the time of decoding of the frame in which the transmission error has occurred to the time of decoding of the subsequent I frame is equal to or greater than the default value (default value) at the receiving terminal, the decoding operation of the decoding unit The inter-frame encoded data cannot be decoded due to the occurrence of the transmission error from the time of decoding of the frame in which the transmission error has occurred until the I frame is subsequently decoded. Operation and so as to decrypt except Tsu portion, controls the decode unit.

Here, the decoding operation that decodes the inter-screen encoded data except for the portion that cannot be decoded due to the occurrence of the transmission error is a decoding operation that performs only the decoding processing for the frames other than the frame in which the transmission error has occurred. is there.
Note that when the image data of each frame is packetized for each data unit smaller than the frame as shown in FIG. 6B, a decoding operation for decoding frames other than the frame in which the transmission error has occurred. In the received image data, packets other than the packet in which the transmission error has occurred may be decoded.

  Furthermore, although the case where the data transmitted from the server to the receiving terminal is video data has been described in the fifth embodiment, the transmission data may be audio data or text data. Even when text data is transmitted by RTP / UDP / IP, the same effect as in the fifth embodiment can be obtained.

  In the second to fourth embodiments, as a data reproducing apparatus that requests image data from a server based on user settings at a terminal and reproduces image data transmitted in response to the request, the Internet or the like is used. In the fifth embodiment, in accordance with the magnitude relationship between the value of the I frame period of the received image data and the default value set in the receiving terminal, FIG. The receiving terminal that switches the decoding operation when an error occurs is shown. Specific examples of the receiving terminal according to the second to fifth embodiments include a PC (personal computer) and the receiving terminal according to the first embodiment. Specific examples include a cellular phone.

(Embodiment 6)
Hereinafter, as a sixth embodiment of the present invention, a mobile phone that requests image data having error tolerance strength specified by a user setting from a server will be described as in the data reproduction device of the second embodiment.
FIG. 16 is a diagram for explaining the mobile phone according to the sixth embodiment.
The cellular phone 300 according to the fifth embodiment outputs a signal processing unit 302 that performs various signal processing and a radio signal N received by the antenna 301 to the signal processing unit 302 as a received signal, and And a wireless communication unit 303 that transmits the transmission signal generated in this way from the antenna 301 as a wireless signal N.

  The mobile phone 300 is processed by a liquid crystal panel (LCD) 306 for displaying an image, a microphone 308 for inputting audio, a speaker 307 for reproducing an audio signal, and the signal processing unit 302. Upon receiving the image signal, the liquid crystal display unit (LCD) 306 controls the display control unit 304 to perform image display based on the image signal, and outputs the input audio signal from the microphone 308 to the signal processing unit 302. And an audio input / output unit 305 that outputs the audio signal processed by the signal processing unit 302 to the speaker 307. Here, for simplicity of explanation, the button operation unit of the mobile phone is not shown.

  Here, the signal processing unit 302 performs the same data reproduction processing as that of the data reproduction device 200b of the second embodiment. That is, the signal processing unit 302 includes an HTTP transmission / reception unit 211, an RTSP message transmission / reception unit 214, an SMIL data analysis unit 212b, an RTP data reception unit 216b, a decoding unit 210, and a user operation unit on the receiving terminal side of the second embodiment. A portion corresponding to 213 is included. The display control unit 304 and the liquid crystal panel (LCD) 306 in the mobile phone 300 of the sixth embodiment correspond to the display unit 218 of the second embodiment.

  In the mobile phone 300 having such a configuration, when an error tolerance strength for image data to be received is set by the user and an operation for reproducing image data corresponding to specific content is performed, the server The video stream suitable for the user set value of the error resilience strength is sequentially transmitted by the RTP packet, and the cellular phone reproduces the video stream from the server and responds to the transmission error occurrence rate during reception of the video stream. Thus, processing for switching the video stream is performed.

  In the sixth embodiment, a mobile phone that performs the same data reproduction processing as that of the data reproduction device of the second embodiment has been described. However, this mobile phone uses the data of the third to fifth embodiments described above. The same data reproduction processing as the data reproduction devices (reception terminals) 200c, 200d, and 200e in the transmission system may be performed.

  Further, in each of the above embodiments, the data reproducing device (receiving terminal) or the data transmitting device (server) is realized by hardware. However, these devices may be realized by software. In this case, the data reproduction apparatus (reception terminal) and the data transmission are recorded by recording a program for performing the data reproduction process or the data transmission process described in the above embodiments in a data storage medium such as a flexible disk. A device (server) can be constructed in an independent computer system.

FIG. 17 is a diagram for explaining a recording medium storing a program for performing data reproduction processing or data transmission processing according to the above-described embodiments by software, and a computer system including the recording medium.
FIG. 17A shows an appearance, a sectional structure, and a flexible disk main body as viewed from the front of the flexible disk, and FIG. 17B shows an example of a physical format of the flexible disk main body.

The flexible disk FD has a structure in which the flexible disk body D is accommodated in a flexible disk case FC, and a plurality of tracks Tr are concentrically formed on the surface of the flexible disk body D from the outer periphery toward the inner periphery. Each track Tr is divided into 16 sectors Se in the circumferential direction. Therefore, the flexible disk FD storing the program has data as the program recorded in an area (sector) Se allocated on the flexible disk main body D.
FIG. 17C shows a configuration for recording the program on the flexible disk FD and a configuration for performing data reproduction processing or data transmission processing by software using the program stored on the flexible disk FD. ing.

  When recording the program on the flexible disk FD, data as the program is written from the computer system Cs to the flexible disk FD via the flexible disk drive FDD. Further, when the data reproducing apparatus or the data transmitting apparatus is constructed in the computer system Cs by the program recorded on the flexible disk FD, the program is read from the flexible disk FD by the flexible disk drive FDD and loaded into the computer system Cs. .

  In the above description, a flexible disk is shown as the data recording medium. However, an optical disk may be used as the data recording medium, and in this case as well, data reproduction processing or data transmission processing by software is performed as in the case of the flexible disk. be able to. Furthermore, the data recording medium is not limited to the optical disk and the flexible disk, and any card can be used as long as it can record a program, such as an IC card or a ROM cassette. Even when these data recording media are used, Data reproduction processing or data transmission processing by software can be performed as in the case of using the flexible disk or the like.

  INDUSTRIAL APPLICABILITY The present invention is useful as a data playback apparatus and data playback method for performing data playback processing of image data. In particular, on the receiving side of image data, on the receiving side, depending on user preferences and the occurrence of transmission errors. This is useful for performing data reproduction processing that enables switching of transmission error tolerance and video quality of acquired image data.

It is a figure for demonstrating the data transmission system by Embodiment 1 of this invention, and has shown the structure (FIG. (A)) of this system, and the data transmission process (FIG. (B)) in this system. It is a figure which shows an example of the description content of the SMIL file FSD1 used with the data transmission system of the said Embodiment 1. FIG. It is a figure which shows the detailed structure of the server 100a and the client terminal 200a which comprise the data transmission system of the said Embodiment 1. FIG. It is a figure explaining the setting method of the concrete error tolerance strength in the receiving terminal 200a of the said Embodiment 1, the method (FIG. (A)) which selects one of two error tolerance strength, and error tolerance by a slide bar A method of specifying the intensity (FIG. (B)) is shown. The description content (FIG. (A)) of the SMIL file FSD2 different from the SMIL file shown in FIG. 2 used in the data transmission system of the first embodiment and the concrete of the video element based on the user setting value Xus2 It is a figure which shows the method (FIG. (B)) of selection. As another example of the plurality of pieces of image data having different error tolerances in the first embodiment, a video stream (FIG. (A)) in which one frame is one video packet, and a video stream (FIG. 1) in which one frame is three video packets. (B)). It is a figure for demonstrating the data transmission system by Embodiment 2 of this invention, and has shown the detailed structure of the server and client terminal which comprise the said system. The figure which shows the table (figure (b)) which correlates the memory content (figure (a)) in the work memory corresponding to the description information of the SMIL file FSD2 used in the second embodiment, and the error occurrence rate and the error tolerance strength It is. It is a figure which shows the example of the exchange of RTSP message at the time of switching the video stream in the said Embodiment 2. FIG. FIG. 10 is a diagram illustrating RTP packets (FIGS. (A) and (b)) stored in reception buffers corresponding to video streams before and after switching when switching video streams in the second embodiment. . It is a figure which shows the flow of the switching process of the video stream in the receiving terminal in the said Embodiment 2. FIG. FIG. 10 is a schematic diagram specifically showing processing performed by the RTSP message transmission / reception unit 214 and the packet RTP data reception unit 216b of the receiving terminal when the video stream is switched in the second embodiment according to the display time. A description of the SMIL file (FIG. (A)) indicating information relating to video streams with different transmission protocols used in the second embodiment, the contents stored in the work memory (FIG. (B)), and an error corresponding to the description. It is a figure which shows the table (FIG. (C)) which correlates an incidence rate and a protocol. It is a figure for demonstrating the data transmission system by Embodiment 3 of this invention, and has shown the detailed structure of the server and client terminal which comprise the said system. It is a figure for demonstrating the data transmission system by Embodiment 4 of this invention, and has shown the detailed structure of the server and client terminal which comprise the said system. It is a figure for demonstrating the mobile telephone as a data reproduction apparatus by Embodiment 6 of this invention. A data storage medium (FIGS. (A) and (b)) storing a program for performing data reproduction processing and data transmission processing of each of the above embodiments by a computer system, and the computer system (FIG. (C)) will be described. It is a figure for doing. It is a figure for demonstrating the communication system for distributing image data using the internet. It is a figure for demonstrating the conventional image coding apparatus, The structure (FIG. (A)) of this image coding apparatus, and the encoding process (FIG. (B)) in the VOP unit in this image coding apparatus are shown. ing. It is a block diagram for demonstrating the conventional image decoding apparatus. It is a figure for demonstrating the data transmission system by Embodiment 5 of this invention, Fig.21 (a) shows the structure of this system, FIG.21 (b) has shown the data transmission process in this system. . It is a figure which shows the detailed structure of the server 100e and the client terminal 200e which comprise the system of the said Embodiment 5. FIG.

Explanation of symbols

10a, 10b, 10c, 10d, 10e Network system 11 Network 21 Button operation unit 21a-21d Cursor key 21e Confirm button 22a Radio wave intensity display screen 22b, 22d Error tolerance setting screen 22c, 22e Operation guidance screen 100a, 100c Server 101 HTTP transmission Means 102 RTSP message reception means 103 RTP data transmission means 104, 219 RTCP report transmission / reception means 110a, 110c, 100e Transmission device 120 Data storage unit 200a, 200b, 200c, 200d, 200e Receiving terminal 201a, 201b Portable terminal 211 HTTP reception means 212 212b, 212e SMIL data analysis means 213 User operation section 214 RTSP message reception means 216, 216b, 216c RTP data receiving means 210, 210d, 210e Decoding unit 218, 218d, 218e Display unit 220, 220e Control unit 300 Mobile phone 301 Antenna 302 Signal processing unit 303 Wireless communication unit 304 Display control unit 305 Audio input / output unit 306 Liquid crystal panel (LCD) )
307 Speaker 308 Microphone Cs Computer system FD Flexible disk FDD Flexible disk drive FSD1-FSD3 SMIL file

Claims (8)

A data playback device for switching and displaying a plurality of video streams,
A receiving unit that receives an RTP packet constituting the video stream, extracts an RTP time stamp from the RTP packet, and calculates a display time on the data reproduction apparatus corresponding to the RTP time stamp;
An error detection unit for detecting an error from an RTP packet constituting the first video stream being received;
A transmission / reception unit for requesting transmission of the second video stream when an error is detected by the error detection unit;
An analysis unit for analyzing the display time calculated by the reception unit;
A control unit for switching a video stream to be displayed,
The receiving unit receives an RTP packet constituting the first video stream and an RTP packet constituting the second video stream after a transmission request for the second video stream is made, Calculating a first display time from an RTP timestamp of the first video stream, and calculating a second display time from an RTP timestamp of the second video stream;
The analysis unit compares the first display time with the second display time to determine whether the second display time is greater than the first display time,
The control unit switches the video stream to be displayed to the second video stream when the second display time is greater than the first display time.
A data reproducing apparatus characterized by that. The data reproducing apparatus according to claim 1, wherein
The transmission / reception unit requests to stop transmission of the first video stream when the second display time is greater than the first display time;
A data reproducing apparatus characterized by that. The data reproducing apparatus according to claim 1, wherein
The receiver stops receiving RTP packets constituting the first video stream when the second display time is greater than the first display time;
A data reproducing apparatus characterized by that. The data reproducing apparatus according to claim 1, wherein
The receiving unit discards RTP packets constituting the first video stream after the second display time when the second display time is greater than the first display time;
A data reproducing apparatus characterized by that. A data reproduction method for switching and displaying a plurality of video streams,
Receiving RTP packets constituting the video stream;
Detecting an error from the RTP packets constituting the first video stream being received;
Requesting transmission of a second video stream when the error is detected;
Receiving an RTP packet constituting the first video stream and an RTP packet constituting the second video stream after a transmission request for the second video stream is made;
An RTP time stamp is extracted from an RTP packet constituting the first video stream, a first display time on the data reproduction apparatus corresponding to the RTP time stamp is calculated, and the second video stream is Extracting an RTP time stamp from the constituting RTP packet and calculating a second display time on the data reproducing apparatus corresponding to the RTP time stamp;
Comparing the first display time with the second display time to determine whether the second display time is greater than the first display time;
Switching the video stream to be displayed to the second video stream when the second display time is greater than the first display time.
A data reproduction method characterized by the above. A data reproduction program for performing data reproduction processing for receiving and displaying a plurality of video streams by a computer,
Receiving RTP packets constituting the video stream;
Detecting an error from the RTP packets constituting the first video stream being received;
Requesting transmission of a second video stream when the error is detected;
Receiving an RTP packet constituting the first video stream and an RTP packet constituting the second video stream after a transmission request for the second video stream is made;
An RTP time stamp is extracted from an RTP packet constituting the first video stream, a first display time on the data reproduction apparatus corresponding to the RTP time stamp is calculated, and the second video stream is Extracting an RTP time stamp from the constituting RTP packet and calculating a second display time on the data reproducing apparatus corresponding to the RTP time stamp;
Comparing the first display time with the second display time to determine whether the second display time is greater than the first display time;
Switching the video stream to be displayed to the second video stream when the second display time is greater than the first display time.
A data reproduction program characterized by the above. A data recording medium for storing a data reproduction program for performing data reproduction processing for receiving and displaying a plurality of video streams by a computer,
The data reproduction program is
Receiving RTP packets constituting the video stream;
Detecting an error from the RTP packets constituting the first video stream being received;
Requesting transmission of a second video stream when the error is detected;
Receiving an RTP packet constituting the first video stream and an RTP packet constituting the second video stream after a transmission request for the second video stream is made;
An RTP time stamp is extracted from an RTP packet constituting the first video stream, a first display time on the data reproduction apparatus corresponding to the RTP time stamp is calculated, and the second video stream is Extracting an RTP time stamp from the constituting RTP packet and calculating a second display time on the data reproducing apparatus corresponding to the RTP time stamp;
Comparing the first display time with the second display time to determine whether the second display time is greater than the first display time;
Switching the video stream to be displayed to the second video stream when the second display time is greater than the first display time.
A data recording medium characterized by the above. The data reproducing apparatus according to claim 1, wherein
The data reproduction device is a mobile phone.
A data reproducing apparatus characterized by that.

Images