JP2001197493A - Method and device for decoding animation and program recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent the deterioration of image quality of a decoded image from propagating by restarting a decoding processing from VOP data of a preceding VOP, when VOP data of the preceding VOP where an early VOP display time is set is inputted after VOP data of a succeeding VOP, where a late VOP display time is set in a decoding unit. SOLUTION: This device is provided with a buffer memory 206 for storing VOP data and a comparing part 204 for comparing the VOP display time of a reception VOP with the VOP display time Td' of VOP data, which is finally outputted to the decoding unit 102 from the buffer memory 206. When VOP data of the preceding VOP is delayed to be after VOP data of the succedding VOP and received after the transmission time, a rearranging unit 101 outputs VOP data from the preceding VOP until a display object VOP to be displayed at this point of time to the decoding unit 2 as a video stream VOPstr.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture decoding method, a moving picture decoding apparatus, and a program recording medium, and more particularly, to transmitting a coded stream obtained by coding moving picture data via a communication line. In this case, a video decoding method and a video decoding apparatus, which reduce the adverse effect of a stream delay due to stream retransmission or the like on a communication line error on a decoded image, and a video decoding method, The present invention relates to a program recording medium storing an image decoding program to be realized by software.

[0002]

2. Description of the Related Art In recent years, a multimedia era has been entered in which voice, images, and other expression media are integrated, and conventional information media, that is, newspapers, magazines, televisions, radios,
Means of transmitting information such as telephones to humans have been taken up as a target of multimedia. Generally, multimedia means not only characters but also graphics, sounds, and especially images, etc. are simultaneously associated with each other. However, in order to target the above-mentioned conventional information media as multimedia, the information must be expressed in a digital format. Representation is an essential condition.

However, when the information amount of each information medium is estimated as a digital information amount, the information amount of a character (per character) is 1 to 2 bytes, while that of telephone quality voice is 1 byte. 64Kbits per second,
Furthermore, a moving image of the current television reception quality requires an information amount of 100 Mbits or more per second, and it is not realistic to handle a huge amount of digital information as it is in the information media such as the telephone and the television. For example, a videophone is an integrated services digital network (ISDN) having a transmission speed of 64 kbps to 1.5 Mbps.
Digital Network) has already been put to practical use, but it is impossible to directly transmit video information obtained by a television camera as digital data via ISDN.

Therefore, what is required is an information compression technique. For example, in the case of a videophone, ITU-T
(International Telecommunication Union Telecommunication Standardization Sector) 261 and H.E. H.263 video compression technology is used. In addition, according to the information compression technology of the MPEG-1 standard, it is possible to store image information together with audio information on a normal music CD (compact disk).

Here, MPEG (Moving Picture Exper)
ts Group) is an international standard for compression processing of moving image data. MPEG-1 defines moving image data at 1.5 Mbps.
This is a standard for compressing information of a television signal to about 1/100. In addition, since the transmission speed for the MPEG-1 standard is mainly limited to about 1.5 Mbps, in MPEG-2 standardized to meet the demand for higher image quality, moving image data is 2 to 15 Mbps.
compressed to s.

Further, at present, MPEG-1, MPEG
-2 and the working group (ISO / IEC JTC1 / SC29 / WG11), which has been working on the standardization of video data compression processing, have made it possible to encode and operate image data on an object basis, MPEG-4 which realizes necessary new functions is being standardized. MPEG-4 initially aimed to standardize low-bit-rate encoding, but now standardization has been extended to higher-bit-rate, more general-purpose encoding corresponding to interlaced images. ing.

By the way, one of the features of MPEG-4 is that
There is a mechanism for simultaneously encoding and transmitting image data corresponding to a plurality of image sequences (objects). This mechanism makes it possible to compose one moving image by combining a plurality of objects. In this mechanism, for example, the foreground and the background can be treated as different image sequences (objects), and the frame frequency, image quality, bit rate, and the like can be individually changed for each image sequence. In addition, a plurality of image sequences are arranged and displayed in a horizontal or vertical direction on a display area like a multi-screen, so that a user can extract only a desired image sequence or enlarge and display it.

As an encoding process for an object having no shape such as a background, an encoding process of encoding only a pixel value signal representing a pixel value of an object as image data, similarly to MPEG-2. It is. On the other hand, as an encoding process for an object having a shape such as a foreground, not only a pixel value signal representing a pixel value of the object but also a shape signal representing the shape of the object is simultaneously encoded as image data. Will be

In MPEG-2 and MPEG-4,
The encoding process of each image sequence is performed by a screen (frame in MPEG-2, VO in MPEG-4)
P), and the encoding process for some screens includes image data corresponding to the screen (target screen) that is the target of the encoding process and the encoding process is completed. A difference encoding process (inter-frame prediction encoding process) for encoding difference data from prediction data obtained from image data corresponding to a screen (processed screen) is performed.

However, since an object having a shape changes its shape and size every moment, MPEG
In the encoding process corresponding to -4, the encoding algorithm of the shape signal and the pixel value signal greatly depends on the object shape of the processed screen. For this reason, the encoding process compatible with MPEG-4 is more vulnerable to transmission errors in the encoded stream than the one that handles image data in which the shape and size of the screen does not change, such as the encoding process compatible with MPEG-2. Further, it is also difficult to perform image restoration using image correlation or image processing using image processing. In other words, MPE
In a G-4 compatible decoding processing system, if an error occurs in the transmission path of an encoded stream, a large deterioration in image quality of a decoded image occurs.

In MPEG-4, as described above, 1
One moving image can be composed of a plurality of image sequences (objects), and each screen of one moving image is a frame.
Each screen of one image series is a VOP (Video Object Plane)
However, when one moving image is composed of one image sequence, the frame and the VOP match.

Hereinafter, a decoding processing system compatible with MPEG-4 will be specifically described. Generally, in a data transmission path (communication line), a data transmission delay occurs due to data retransmission or the like as a measure against noise or congestion. Therefore, a moving image encoding apparatus that encodes image data and outputs an encoded stream, The coded stream transmitted from the moving image transmission device or the like that transmits the coded stream has the bit order (stream order at the time of transmission) constituting the coded stream in the stream order (stream order at the time of reception) on the receiving side. (Stream order at the time of reception). Specifically, the order of streams (hereinafter referred to as VOP data) corresponding to some VOPs in the encoded stream differs between the order at the time of transmission and the order at the time of reception.

Therefore, on the transmission side of the encoded stream,
For example, when an encoded stream is subdivided and transmitted in transmission units called packets, number information indicating the transmission order is added to each packet. This allows the receiving side of the encoded stream to rearrange the VOP data in the received encoded stream based on the number information given to each packet so that the order matches the transmission order.

FIG. 30 is a block diagram for explaining a conventional moving picture decoding apparatus. This video decoding device 1
000 receives encoded streams Vin corresponding to a plurality of objects constituting one moving image, performs a process of rearranging the VOP data with respect to the encoded streams corresponding to the individual objects, and then performs encoding. It performs decoding processing on the stream Vin, synthesis of decoded image data of each object, and the like. Also, the encoded stream V of each object
For each VOP data in “in”, information indicating a time at which the VOP data should be displayed on the receiving side (VOP display time) is added on the transmitting side.

That is, the moving picture decoding apparatus 1000
Encoded stream V corresponding to a predetermined object from the transmitting side
in, and performs a process of rearranging the stream (VOP data) corresponding to each VOP in the encoded stream so that its order matches the order of the VOP data on the transmission side. Rearrangement unit 1001 for outputting the converted stream Vin as a video stream VOPstr, and the video stream VO
For Pstr, reference image data VOPref as required
To the decoded image data VOPou
and a decoding unit 1002 for creating t.
Also, the moving picture decoding apparatus 1000 records the decoded picture data VOPout from the decoding unit 1002, and the picture of the recorded decoded picture data VOPout required for the decoding process. It has a video memory 1003 for supplying data to the decoding unit 1002 as the reference screen data VOPref.

Here, the reordering unit 1001 has a built-in buffer memory 1101 for temporarily storing the encoded stream Vin from the transmitting side and a timer, and forms a moving image specified by the receiving side. A display time output unit 1102 for generating a display time (designated display time) Td of each screen, and a specific VO based on the designated display time Td.
The P data is output from the buffer memory 1101 to the decoding unit 1002 as a video stream VOPstr. The specific VOP data is VOP data stored as an encoded stream Vin in the buffer memory 1101, and the VOP display time is the VOP display time of the VOP data already output to the decoding unit as the video stream VOPstr. The VOP data is later and earlier than the designated display time Td. The decoding unit 1002 performs a VOP on which an intra-screen encoding process using a pixel value correlation within the screen is performed.
Data is subjected to a decoding process that does not refer to the reference image data VOPref, and VOP data that has been subjected to an inter-screen predictive encoding process using a pixel value correlation between the screens,
It has a decoder 1103 that performs decoding processing with reference to the reference image data VOPref to generate decoded image data VOPout of each VOP constituting the corresponding object.

In FIG. 30, the sorting unit 10
01, decoding unit 1002, and video memory 10
03, only the one corresponding to one object is shown, but the moving picture decoding apparatus 1000 includes at least the reordering unit 1001, the decoding unit 1002, and the video memory 1003, each of which at least constitutes the moving picture. Have the maximum number of objects displayed at the same time. The maximum number of objects displayed simultaneously is equal to or less than the number of all objects appearing in one moving image. Also, the rearrangement units in the video decoding device are all displayed time output units 110.
It is not necessary that the rearrangement unit corresponding to at least one object has the display time output unit 1102. In this case, the display time output unit 1
The buffer memory 1 of the reordering unit not having 102
The designated display time Td is supplied to 101 from the rearrangement unit having the display time output unit 1102.

Further, the moving picture decoding apparatus 1000 generates a display screen selection signal VOPsel for selecting the decoded picture data corresponding to the VOP whose VOP display time is closest to the specified display time Td based on the specified display time Td. Is output to the video memory corresponding to each object, and the VOP display time is changed from the video memory corresponding to each object to the designated display time Td.
The video compositor unit 1004 reads the decoded image data corresponding to the VOP closest to the VOP as display image data VOPdsp, and combines the display image data read from the video memory corresponding to each VOP to create the combined image data Vout. And a display unit 1005 for displaying an image of one scene forming a moving image based on the composite image data Vout. Display unit 100
In 5, the process of displaying an image of one scene at the designated display time Td is repeatedly performed, so that the moving image is displayed.

Next, the operation of the conventional moving picture decoding apparatus will be described. When the coded streams Vin corresponding to a plurality of objects constituting one moving image are input to the moving image decoding apparatus 1000, the coded streams Vin corresponding to the respective objects are reordered according to the respective objects. The data is sequentially stored in the buffer memory of the unit. Note that the rearrangement process for the encoded stream, the decoding process for the encoded stream, and the process for storing the decoded image data in the video memory are common to each object. explain.

In the buffer memory 1101, all the input coded streams are stored until the stored data amount reaches the buffer capacity. Thereafter, when the coded stream is input, the coded stream stored in the buffer memory 1101 is changed from the earlier input stream to the buffer memory by an amount corresponding to the data amount of the input coded stream. Deleted in order.

A display time output section 1102 having a built-in timer generates a display time (designated display time) Td of each screen constituting a moving image on the receiving side, and stores information indicating the designated display time Td in a buffer memory. Output to 1101. In the buffer memory 1101, the designated display time T
d, predetermined VOP data stored in the buffer memory 1101 is selected, and the selected VOP data is selected.
The data is output to the decoding unit 1002 as a video stream VOPstr.

The predetermined VOP data selected at this time is such that the VOP display time is earlier than the designated display time Td,
At this point, the V of the last VOP data among the VOP data already output to the decoding unit 1002
It is later than the OP display time. Accordingly, in rearrangement unit 1001, the order of VOP data in the received encoded stream (hereinafter, also referred to as receiving-side VOP order) changes the order of VOP data in the encoded stream output from the transmitting side (transmitting side). VOP data is rearranged so as to match the VOP order. In other words, the transmission-side VOP order is different from the reception-side VOP order depending on conditions such as the communication path, and the VOP data is stored in the buffer memory 1 in a different order from the transmission-side VOP order.
101, the buffer memory 1101
After that, the VOP data is output to the decoding unit 1002 as the video stream VOPstr in the correct VOP order that matches the transmission side VOP order.

Then, in the decoding unit 1002, the input video stream VOPstr is
3 and each V
OP decoded image data VOPout is output. Then, the decoded image data VOPout output from the decoding unit 1002 is temporarily stored in the video memory 1003. Here, the input video stream VOPstr
Is VOP data corresponding to an intra VOP that has been subjected to intra-frame encoding processing, the decoding unit 100
In 2, the video stream VOPstr is subjected to an intra-screen decoding process that does not refer to the decoded data stored in the bit memory 1003. On the other hand, when the input video stream VOPstr is VOP data corresponding to the inter VOP subjected to the inter-picture predictive coding processing, the decoding unit 1002 stores the video stream VOPstr in the video memory 10.
Inter-picture predictive decoding is performed with reference to the decoded data (reference image data) VOPref stored in the image data 03.

The information indicating the designated display time Td from the display time output unit 1006 is also supplied to the video composite unit 1004. In the video composite unit 1004, based on the specified display time Td, a display screen selection signal VOPsel for selecting the decoded image data of the VOP whose VOP display time is closest to the specified display time Td is output to the video memory 1003. .
Then, from the video memory 1003, the designated display time T
The decoded image data of the VOP in which the VOP display time closest to d is set is supplied to the video composite unit 1004 as the display image data VOPdsp. Here, as the display image data VOPdsp, data corresponding to each object is supplied from the video memory corresponding to each object to the video composite unit 1004.

The video composite unit 100
4 is the video memory 1
When the decoded image data stored in the video memory 1003 has already been output to the video composite unit 1004 at the time when the decoded image data is input to the video memory 1003, or when the decoded image data is not stored in the video memory 1003, Does not output decoded image data in response to the display screen selection signal VOPsel from the video composite unit 1004.

The video composite unit 10
At 04, the combining process is performed on the display image data VOPdsp of each object, and the combined image data Vout obtained by the combining process is output to the display unit 1005. The composite image data Vout is image data corresponding to one scene (one screen) of a moving image. Display unit 100
In 5, the image of each scene of the moving image is displayed based on the composite image data Vout. The display unit 100
In 5, the process of displaying the image of each scene based on the designated display time Td is repeatedly performed, so that the moving image is displayed.

FIG. 31 is a diagram for explaining the flow of processing of an encoded stream in a conventional moving picture decoding apparatus. Here, a process for an encoded stream of one image sequence (object) is shown. First, as shown in FIG. 31 (a), the VOP in the received encoded stream
A case where the data order is the correct order that matches the VOP order on the transmission side will be described.

As described above, when there is no loss of the coded stream due to the transmission path delay, the VOP data of, for example, VOP1 to VOP4 is used as the coded stream Vin, and the VOP display times T1 to T
4 (T1 <T2 <T3 <T4).

When these VOP data are inputted to the buffer memory 1101 of the rearrangement unit 1001 and a predetermined time (delay time Bdt1) has elapsed, based on the designated display time Td from the display time output unit 1102. ,
The VOP data of VOP1 to VOP4 are decoded as video streams VOPstr1 to VOPstr4 by the decoding unit 1002.
Is output to VOP in the buffer memory 1101
The data delay time Bdt1 is determined by the buffer memory 110
The delay time of the VOP data is larger as the capacity of the buffer memory 1101 is larger. In this case, the capacity of the buffer memory 1101 is a capacity capable of storing VOP data corresponding to two VOPs.

In the decoding unit 1002, the video streams VOPstr1 to VOPstr4 of each VOP are
The decoding is performed in the order of the OP display times T1 to T4. Each VO
The decoded image data of P is temporarily recorded in the video memory 1003, and the decoded image data of each VOP is converted into the display image data VOPdsp1 to VOPdsp4 in accordance with the display screen selection signal VOPsel from the video compositor unit 1004. 1004.

Next, a case where the order of VOP data in the received coded stream does not match the order of VOPs on the transmitting side as shown in FIG. 31 (b) will be described. When the order of the VOP streams in the encoded stream is not correct, the rearrangement unit 1001 of the video decoding apparatus 1000
VOP data of 1 to VOP4 are encoded streams Vin
Are input in the order in which they arrived at the video decoding apparatus 1000, that is, in the order of VOP1, VOP3, VOP2, and VOP4.

These VOP data are input to the buffer memory 1101 of the rearrangement unit 1001 and when a predetermined time (delay time Bdt2) has elapsed, based on the designated display time Td from the display time output unit 1102. ,
The VOP data of VOP1 to VOP4 are decoded as video streams VOPstr1 to VOPstr4 by the decoding unit 1002.
Is output to At this time, when the VOP data of VOP2 is output from the rearrangement unit 1001 based on the designated display time Td, the VOP data of VOP2 is stored in the buffer memory 1101.
The VOP data of VOP2 arriving at the video decoding device 1000 after the VOP data of VOP3 is output to the decoding unit 1002 before the VOP data of VOP3. Here, the capacity of the buffer memory 1101 is a capacity capable of storing VOP data corresponding to three VOPs.

In the decoding unit 1002, the video streams VOPstr1 to VOPstr4 of each VOP are
The decoding is performed in the order of the OP display times T1 to T4. Each VO
The decoded image data of P is temporarily recorded in the video memory 1003, and the decoded image data of each VOP is converted into the display image data VOPdsp1 to VOPdsp4 in accordance with the display screen selection signal VOPsel from the video compositor unit 1004. 1004.

[0034]

However, since the amount of VOP data that can be stored in the buffer memory 1101 of the reordering unit 1001 has a certain limit, the transmission path delay causes the arrival order of the VOP data. There is no problem if the disturbance of the transmission order can be absorbed by the reordering unit 1001, but if a transmission line delay large enough to prevent the disturbance of the arrival order occurs, it will greatly hinder the decoding operation.

FIG. 32 is a diagram for explaining the flow of processing of an encoded stream in a conventional moving picture decoding apparatus. Here, the VOP data of VOP2 is replaced by another VOP.
Of the VOP data of
The figure shows a case where the OP data arrives at the video decoding device with a delay that cannot be rearranged. In other words,
In this case, the arrival time of the VOP data of VOP2 is later than the original arrival time by the delay time in the buffer memory 1101 or more.

When the VOP data is input to the reordering unit 1001 at the correct input timing, the VOP data is output from the reordering unit 1001 to the decoding unit 1002 at the output timing after the buffer delay time has elapsed from the correct input timing. Will be output. For this reason, the VOP data is slightly delayed,
0, the VOP data arriving late is not received by the reordering unit 100 before the output timing.
If it is input to 1, it will be output to the decoding unit 1002 normally.

However, as shown in FIG.
If the transmission delay of the VOP data is large, VOP2
When the VOP data of VOP2 is output to the decoding unit 1002 (the output timing of VOP2),
The data is stored in the buffer memory 110 of the reordering unit 1001.
A situation occurs that is not stored in 1. In this case, V
The OP2 VOP data is not decoded by the decoding unit 1002.

Thereafter, at the time when the VOP data of VOP3 is output to the decoding unit 1002 (output timing of VOP3), the VOP data of VOP3 stored in the buffer memory 1101 of the rearrangement unit 1001 is output to the decoding unit 1002. Is done. Decryption unit 10
In 02, decoding of the VOP data of VOP3 is performed following decoding of the VOP data of VOP1.

At the time when the VOP data of VOP4 is output to the decoding unit 1002 (output timing of VOP4), even if the VOP data of VOP2 arrives at the video decoding device 1000, the VOP data of VOP2 is discarded. , VOP4 only VOP data decoding unit 1
002. This is because the decoding order is VOP3
This is because the VOP data of VOP2 that is earlier than the VOP data of VOP3 cannot be decoded after the VOP data of VOP3 is decoded. In the decoding unit 1002, the decoding process of the VOP data of VOP4 is performed following the decoding process of the VOP data of VOP3.

On the other hand, the video memory 1003 stores the VOP1, VOP3, VO from the decoding unit 1002.
The decoded image data of P4 is stored, and at the time when each VOP is to be displayed, the decoded image data of each VOP is output from the video memory 1003 to the video compositor unit 1004 as the display image data VOPdsp. However, since VOP data of VOP2 (video stream VOPstr2) is not input to the decoding unit 1002,
2 is to be displayed, the decoded image data of VOP1 (display image data VO
Pdsp1) is output from the video memory 1003 to the video compositor unit 1004.

In this case, since the decoded image data of VOP2 cannot be referred to at the time of the inter-picture decoding process for the VOP data of VOP3, the image quality of the decoded image of VOP3 is deteriorated. Also, in the inter-picture decoding process for the VOP data of VOP4, the VO with deteriorated image quality is used.
Since the decoded image data of P3 is referred to, the image quality of the decoded image of VOP4 is also deteriorated.

As described above, in the conventional video decoding apparatus, when a part of the coded stream (that is, specific VOP data) is received with a considerable delay due to a transmission path delay, the VOP data is received with a delay. VOP of VOP
There is a problem that the image quality of decoded images of all VOPs whose VOP display time is later than the display time is deteriorated.

In the conventional moving picture decoding apparatus 1000, the memory capacity of the buffer memory 1101 of the reordering unit 1001 is increased, and VOP data of each VOP is input to the reordering unit 1001 as an encoded stream Vin. By increasing the time until the corresponding VOP data is output from the reordering unit 1001 as the video stream VOPstr, the VOP data of the VOP arriving with a significant delay is output as the video stream VOPstr in a normal order. It becomes possible.

In this case, however, the delay time from when the VOP data of each VOP is input to the video decoding device 1000 as the encoded stream Vin until the decoded image of each VOP is displayed is totally growing. As a result, especially in a monitoring system that transmits a captured image to a receiving terminal and displays the image in real time, the function of the monitoring system is impaired due to an increase in response time from input of an encoded stream to display of a decoded image. Problems arise. Further, an increase in the memory capacity of the buffer memory 1101 in the rearrangement unit 1001 also causes a problem that power consumption and equipment cost increase as it is.

The present invention has been made in view of the above-described problems. Even when a part of a coded stream obtained by coding moving image data is received with a considerable delay, the present invention is not limited to this. A moving image decoding method and a moving image decoding apparatus, and a moving image decoding method capable of preventing deterioration of image quality of a decoded image corresponding to a received portion from being successively passed on to decoded images displayed thereafter. It is an object of the present invention to provide a program recording medium storing a program for performing a decoding method.

[0046]

According to a first aspect of the present invention, there is provided a moving image decoding method comprising the steps of: generating a coded stream obtained by coding image data of a moving image composed of images for which individual times are set; A moving image decoding method for decoding, comprising: receiving encoded data of each image as an encoded stream from a transmission side; and transmitting encoded data of each received image to an individual time set in each image. A rearrangement step of performing a rearrangement process of sending out images at a transmission time corresponding to each image in order from an earlier one, and performing a decoding process on the encoded data of each image transmitted by the rearrangement process. A decoding step of generating decoded image data corresponding to the image, wherein the rearranging step includes the step of: converting encoded data of an image to be sent at the first sending time to a second sending time later than the first sending time. Yo When receiving late, in which the individual time has the steps of sending an image of coded data which is time within the range from the first sending time to the second sending time.

According to a second aspect of the present invention, in the moving picture decoding method according to the first aspect, the rearranging step includes displaying encoded data of an image forming the moving image by displaying encoded data of the moving image. If the last image in the order is received before the time to be displayed, the coded data of the received image should be sent out, and the coded data of the image forming the moving image should be displayed. This is a step of discarding the encoded data of the received image when received after the time.

According to a third aspect of the present invention, in the moving picture decoding method according to the first aspect, the rearranging step includes the step of: transmitting encoded data of an image to be transmitted at the first transmission time.
When the received data is received later than the second transmission time that is later than the first transmission time, the coded data of all the images whose individual time is within the range from the first transmission time to the second transmission time is received. If the transmission is possible, the encoded data of all the images is transmitted, and the encoded data of all the images whose individual time is within the range from the first transmission time to the second transmission time is not transmitted. If possible, the step is a step of discarding the encoded data of the received image.

[0049] The moving picture decoding method according to the present invention (claim 4) decodes a coded stream obtained by coding moving picture image data composed of pictures each having an individual time set. Coded data of each image is received as a coded stream from the transmission side, and the coded data of each received image is sent out in ascending order of the individual time set for each image. A rearranging step of performing a rearranging process, and performing a decoding process on the encoded data of each image transmitted by the rearranging process,
A decoding step of generating decoded image data corresponding to each image, the decoding step, by the rearrangement process, when the encoded data of an image that can not be normally decoded is transmitted, The decoding process for the coded data of the image is interrupted from the time when the coded data is transmitted to the time when the coded data of the image that can be normally decoded is transmitted by the rearrangement process. It is a step.

According to a fifth aspect of the present invention, in the moving picture decoding method according to the fourth aspect, the decoding step includes transmitting encoded data of an image for which the normal decoding process is impossible. Whether to interrupt the decryption process at the time,
This is a step of determining based on a user's operation signal.

According to a sixth aspect of the present invention, in the moving picture decoding method according to the fourth aspect, the decoding step is such that the encoded data of the image transmitted by the rearrangement processing indicates the shape of the image. Including a shape determination step of determining whether or not the image includes shape information, and when the encoded data of the image includes shape information, when the encoded data of the image that cannot be normally decoded is transmitted. ,
From the point in time at which the encoded data is transmitted, until the point at which the encoded data of the image that can be normally decoded is transmitted, the decoding of the encoded data is interrupted, and the encoded data of the image is interrupted. Does not include the shape information, the decoding of the encoded data is performed regardless of whether the encoded data of the image transmitted by the rearranging process can perform the normal decoding process. This is a step for continuing the conversion process.

A moving picture decoding apparatus according to the present invention (claim 7) decodes a coded stream obtained by coding moving picture image data composed of pictures each having an individual time set. Encoding apparatus, receives the encoded data of each image as an encoded stream from the transmission side, and encodes the received image data of each image in order from the earliest individual time set for each image, A data rearranging unit that outputs at a transmission time corresponding to each image, and performs a decoding process on the encoded data of each image transmitted from the data rearranging unit, and outputs decoded image data corresponding to each image. A decoding unit to generate, and a display image output unit to output decoded image data corresponding to each image output from the decoding unit as display image data so that display of each image is performed at a predetermined timing. When the data rearrangement unit receives the encoded data of the image to be transmitted at the first transmission time later than the second transmission time later than the first transmission time, the individual time is set to the first transmission time. The encoded data of the image which is a time within the range from the transmission time to the second transmission time is transmitted.

According to the present invention (claim 8), in the moving picture decoding apparatus according to claim 7, the display image output unit is changed by the data rearrangement unit so that the individual time is changed from the first transmission time to the first transmission time. When the encoded data of the image whose time is within the range up to the transmission time 2 is transmitted, among the images from which the encoded data is transmitted, the individual time of the image whose individual time is closest to the second transmission time is determined. Only the decoded image data is output as display image data.

[0054] A moving picture decoding apparatus according to the present invention (claim 9) decodes a coded stream obtained by coding moving picture image data composed of pictures for which individual times have been set. Encoding apparatus for receiving encoded data of each image as an encoded stream from a transmission side, and transmitting the encoded data of each received image in ascending order of the individual time set for each image. A data rearranging unit that performs decoding processing on encoded data of each image output from the data rearranging unit to generate decoded image data corresponding to each image; A display image output unit that outputs decoded image data corresponding to each image output from the decoding unit as display image data so that display of each image is performed at a predetermined timing. When the encoded data of an image for which normal decoding cannot be performed is transmitted from the reordering unit, the encoding of the image for which normal decoding can be performed from the data reordering unit after the transmission of the encoded data. The decoding process for the encoded data of the image is interrupted until the encoded data is transmitted.

According to a tenth aspect of the present invention, in the moving picture decoding apparatus according to the ninth aspect, the display image output unit includes:
While the decoding process for the encoded data of the image is interrupted by the decoding unit, the decoded image data subjected to the last decoding process by the decoding unit is output as display image data. It was done.

The program storage medium according to the present invention (claim 11) uses a computer to perform a decoding process on an encoded stream obtained by encoding image data of a moving image composed of images for which individual times have been set. A program recording medium storing the program, wherein the computer receives the encoded data of each image as an encoded stream from a transmission side, and converts the received encoded data of each image into each image. Is performed in order from the earliest individual time set in the transmission time corresponding to each image. In this rearrangement processing, the computer encodes the image to be transmitted in the first transmission time. If the data is received later than the second transmission time later than the first transmission time, the computer sets the individual time to the first transmission time. A rearrangement program for transmitting encoded data of an image having a time within the range from the time to the second transmission time, and a computer decoding the encoded data of each image transmitted by the rearrangement processing And a decoding program for performing decoding processing to generate decoded image data corresponding to each image.

According to the program storage medium of the present invention (claim 12), a computer performs decoding processing on an encoded stream obtained by encoding image data of a moving image composed of images each having an individual time set. A program storage medium storing the program, wherein the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image into each image. A rearrangement program for performing a rearrangement process of sequentially sending out the earliest set individual time, and a computer performing a decoding process on the encoded data of each image transmitted by the rearrangement process. To generate decoded image data corresponding to each image, and in the decoding process, the normal When the encoded data of the impossible image is transmitted, from the transmission time of the encoded data to the subsequent transmission of the encoded data of the image that can be normally decoded by the rearrangement process. And a decoding program for causing the computer to interrupt the decoding process on the encoded data of the image.

A moving picture decoding method according to the present invention (claim 13) decodes a coded stream obtained by coding picture data of a moving picture composed of pictures for which individual times have been set. Encoding method, receives the encoded data of each image as an encoded stream from the transmission side, the encoded data of each received image, in order from the earliest individual time set for each image, A reordering step of performing a reordering process of sending out the image at a transmission time corresponding to each image; and a decoding process of performing a decoding process on the encoded data of each image transmitted by the above-described reordering process. And a decoding step of generating data. The reordering step includes the step of, if the encoded data of the image to be sent at the required sending time cannot be received by the required sending time, Until the encoded data of the image to be transmitted at the output time is received, the transmission of the encoded data of the image is stopped, and the encoded data of the image to be transmitted at the required transmission time is received. This is a step of restarting transmission.

[0059] A moving picture decoding method according to the present invention (claim 14) decodes a coded stream obtained by coding moving picture image data composed of pictures each having an individual time set. Encoding method, receives the encoded data of each image as an encoded stream from the transmission side, the encoded data of each received image, in order from the earliest individual time set for each image, A reordering step of performing a reordering process of sending out the image at a transmission time corresponding to each image; and a decoding process for the encoded data of each image sent out by the above-described reordering process to obtain a decoded image corresponding to each image. And a decoding step of generating data. The reordering step includes the step of: if encoded data of an image to be transmitted at a required transmission time cannot be received by the required transmission time, a predetermined condition. Until the condition is satisfied, or until the encoded data of the image to be transmitted at the required transmission time is received, the transmission of the encoded data of the image is stopped, and after the predetermined condition is satisfied, or the required transmission time After receiving the encoded data of the image to be sent to the server, the transmission of the encoded data of the image is restarted.

According to a fifteenth aspect of the present invention, in the moving picture decoding method according to the fourteenth aspect, a buffer memory for storing the coded stream of each image received at the time of the rearrangement processing under the predetermined condition. Of the remaining storage capacity is not more than a predetermined value.

According to a sixteenth aspect of the present invention, in the moving picture decoding method according to the fourteenth aspect, when the predetermined condition is satisfied, the transmission of encoded image data is stopped during the rearrangement process. The condition is such that the time is equal to or greater than a predetermined value.

[0062] A moving picture decoding apparatus according to the present invention (claim 17) decodes a coded stream obtained by coding picture data of a moving picture composed of pictures each having an individual time set. Encoding device, receives the encoded data of each image as an encoded stream from the transmission side, and, from the earliest individual time set for each image, the encoded data of each received image, A data rearranging unit that outputs at a transmission time corresponding to each image, and performs a decoding process on the encoded data of each image output from the data rearranging unit, and decodes the decoded image data corresponding to each image. A decoding unit to generate, and a display image output unit to output decoded image data corresponding to each image output from the decoding unit as display image data so that display of each image is performed at a predetermined timing. If the data rearrangement unit cannot receive encoded data of an image to be transmitted at a required transmission time by the required transmission time, it receives encoded data of an image to be transmitted at the required transmission time. Until the transmission of the encoded data of the image is stopped, the encoded data of the image to be transmitted at the required transmission time is received, and then the transmission of the encoded data of the image is restarted.

A moving picture decoding apparatus according to the present invention (claim 18) decodes a coded stream obtained by coding picture data of a moving picture composed of pictures each having an individual time set. Encoding device, receives the encoded data of each image as an encoded stream from the transmission side, and, from the earliest individual time set for each image, the encoded data of each received image, A data rearranging unit that outputs at a transmission time corresponding to each image, and performs a decoding process on the encoded data of each image output from the data rearranging unit, and decodes the decoded image data corresponding to each image. A decoding unit to generate, and a display image output unit to output decoded image data corresponding to each image output from the decoding unit as display image data so that display of each image is performed at a predetermined timing. The data rearranging unit, if the encoded data of the image to be transmitted at the required transmission time cannot be received by the required transmission time, until a predetermined condition is satisfied, or at the required transmission time. Until the encoded data of the image to be transmitted is received, the transmission of the encoded data of the image is stopped, and after the predetermined condition is satisfied, or the encoded data of the image to be transmitted at the required transmission time is received. Thereafter, transmission of the encoded data of the image is restarted.

According to a nineteenth aspect of the present invention, in the moving picture decoding apparatus according to the eighteenth aspect, the display image output unit is provided with the data rearranging unit for converting the encoded data of the image to the decoding unit. When the transmission of the encoded data of the image is restarted after the transmission is stopped, the display of the image within a certain period is reduced, and the delay of the display timing of the image caused by the stop of the transmission becomes smaller each time the image is displayed. It should be done as follows.

The program storage medium according to the present invention (claim 20) uses a computer to perform a decoding process on an encoded stream obtained by encoding image data of a moving image composed of images for which individual times have been set. A program storage medium storing the program, wherein the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image into each image. The rearrangement process of sending out the images at the transmission time corresponding to each image is performed in order from the earliest individual time set in the computer. If the computer cannot receive the image by the required transmission time, the computer outputs the code of the image to be transmitted at the required transmission time. Until the encoded data of the image is received, stop the transmission of the encoded data of the image, and after receiving the encoded data of the image to be transmitted at the required transmission time, rearrange to restart the transmission of the encoded data of the image. A decoding program configured to perform a decoding process on the encoded data of each image transmitted by the rearrangement process and generate decoded image data corresponding to each image by a computer. It is.

According to the program storage medium of the present invention (claim 21), a computer performs decoding processing on an encoded stream obtained by encoding image data of a moving image composed of images each having an individual time set. A program storage medium storing the program, wherein the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image into each image. The rearrangement process of sending out images at the transmission time corresponding to each image is performed in order from the earliest individual time set in the computer. In the case of the rearrangement process, the computer encodes the image data to be sent out at the required transmission time. Is not received by the required transmission time, the computer waits until a predetermined condition is satisfied, or The transmission of the encoded data of the image is stopped until the encoded data of the image to be transmitted at the required transmission time is received, and the code of the image to be transmitted at the required transmission time after the predetermined condition is satisfied or at the required transmission time. After receiving the encoded data, the rearrangement program for resuming the transmission of the encoded data of the image, and the computer performs a decoding process on the encoded data of each image transmitted by the above-described rearranging process. , And a decoding program for generating decoded image data corresponding to each image.

[0067]

Embodiments of the present invention will be described below. (Embodiment 1) FIG. 1 is a diagram for explaining a video decoding apparatus according to Embodiment 1 of the present invention. The moving picture decoding apparatus 100 according to the first embodiment includes a reordering unit 10 in the conventional moving picture decoding apparatus 1000 shown in FIG.
01 instead of V
If the VOP data of the OP (preceding VOP) is received after the VOP data of the VOP (succeeding VOP) for which the late VOP display time is set, the decoding device specified from the preceding VOP starts. The reordering unit 101 outputs VOP data up to the display target VOP indicated by the designated display time. Here, the display target V
OP is a VOP in which the time before the specified display time and the time closest to the specified display time is set as the VOP display time.
It is. Also, the VOP data of the preceding VOP is
If the VOP data of the preceding VOP is received after the VOP data of the OP, strictly speaking, the transmission time at which the VOP data of the preceding VOP is output from the rearrangement unit 101 (in other words, the decoding by the subsequent units 102 and 104) (The time at which the processing and the synthesis processing are performed) and the second case where the VOP data of the preceding VOP is received later than the transmission time at which it is output from the rearrangement unit 101. There are cases.

However, even when the VOP data of the preceding VOP is received after the VOP data of the succeeding VOP,
In the first case, in which the VOP data of the preceding VOP is received in time for the transmission time, the rearrangement unit 101 performs a normal rearranging process similar to the conventional rearranging process to execute the preceding VOP received later. Is output earlier than the VOP data of the succeeding VOP received earlier. Therefore, in the unit 101, the preceding VO
The reordering process of outputting VOP data from P to the display target VOP indicated by the designated display time designated on the decoding device side (the reordering process unique to the first embodiment) is not performed.

On the other hand, the VOP data of the preceding VOP lags behind the VOP data of the succeeding VOP, and
In the second case where the VOP data of the subsequent VOP is received later than the transmission time, the VOP data of the subsequent VOP is usually transmitted from the rearrangement unit 101. For this reason, the reordering unit 101 uses the preceding VO
A rearrangement process of outputting VOP data from P to the display target VOP indicated by the specified display time specified by the decoding device (a rearrangement process unique to the first embodiment) is performed.

Therefore, in the following description of each embodiment,
The case where the VOP data of the preceding VOP is received after the VOP data of the succeeding VOP is the first case where the normal rearrangement processing is performed (the VOP data of the preceding VOP is received in time for its transmission time). This is not the case, but a second case (a case where the VOP data of the preceding VOP is received later than the transmission time) in which the unique rearrangement process of the first embodiment is performed.

Also, the moving picture decoding apparatus 100 stops the decoding process of the VOP data including the error, instead of the decoding unit 1002 in the conventional moving picture decoding apparatus 1000, and converts the decoded VOP into a decoded VOP. The decoding unit 102 decodes only VOP data of a VOP for which a VOP display time later than the set VOP display time is set.

Further, the moving picture decoding apparatus 100
Like the conventional moving picture decoding apparatus 1000, the moving picture decoding apparatus 1000 has a video memory 103 for recording the decoded screen data VOPout obtained by the decoding processing in the decoding unit 102.

Here, the reordering unit 101, the decoding unit 102, and the video memory 103 each include at least one simultaneously displayed image sequence (at least one moving image) similar to the conventional moving image decoding apparatus 1000. object)
The video memory 103 is provided in the video memory 100 in the conventional moving picture decoding apparatus 1000.
3 has the same configuration.

In addition, the moving picture decoding apparatus 100 starts displaying from the video memory corresponding to each object displayed at that time (only the video memory 103 corresponding to one object is shown in FIG. 1). The decoded image data VOPout of each of the above objects is read out as display image data VOPdsp by the screen selection signal VOPsel, and the combined image data VOPdsp of each object is combined to create composite image data Vout which is image data of one scene of one moving image. And the composite image data Vou
and a display unit 105 for displaying the composite image as one moving image scene based on the t.

Here, the video compositor unit 1
Reference numeral 04 denotes decoded image data stored in the video memory 103 when the display screen selection signal VOPsel from the video composite unit 104 is input to the video memory 103, and the designated display time indicated by the display screen selection signal VOPsel The display unit 10 displays the decoded image data of the VOP in which the VOP display time closest to Td is set.
5 is output.

The display unit 105 in the moving picture decoding apparatus 100 according to the first embodiment has exactly the same configuration as the display unit 1005 in the conventional moving picture decoding apparatus 1000.

Next, the operation will be described. When the coded stream Vin corresponding to a plurality of objects (image sequences) constituting one moving image is received by the moving image decoding apparatus 100 according to the first embodiment, each of the objects displayed at that point The corresponding coded stream Vin is sequentially input to a permutation unit corresponding to each object (only permutation unit 101 corresponding to one object is shown in FIG. 1). In the following description, the rearrangement process for the coded stream, the decoding process for the coded stream, and the process for storing the decoded image data in the video memory are common to each object. Only one object will be described.

In the rearrangement unit 101, when VOP data is input in the order of the VOP display time set in each VOP, each VOP data is converted into the video stream VOPstr in the order input to the rearrangement unit 101.
Is output to the decoding unit 102. On the other hand, in the rearrangement unit 101, the VOP data of the VOP (preceding VOP) for which the early VOP display time is set is changed to the late VO data.
V of VOP (subsequent VOP) for which P display time is set
If the input data is input after the OP data, the VOP data from the preceding VOP to the display target VOP (the VOP display time is before the specified display time and the closest VOP) is set as the video stream VOPstr as the decoding unit 10.
2 is output.

When the video stream VOPstr is input to the decoding unit 102, the video stream VOstr
If the VOP data included in Pstr is normal VOP data having no defect and can be decoded normally, the decoding unit 102 sequentially performs decoding processing on each VOP data in the video stream VOPstr. On the other hand, if the video stream VOPstr includes defective VOP data that is likely to cause image quality degradation in the display image due to, for example, the absence of reference image data, the decoding unit 102 sets the video stream VOPstr.
The decoding process for the defective VOP data in is stopped until normal VOP data is input.

Here, the defective VOP data is obtained by skipping the VOP data for which the early VOP display time is set, and removing defects such as VOP data for which the late VOP display time is input to the decoding unit 102 and for which a transmission error has occurred. VOP data included. In addition, the normal VOP data is input later than the VOP data in which the late VOP display time previously input to the decoding unit 102 is set.
Examples include VOP data for which an early VOP display time is set, VOP data of a VOP (intra-VOP) subjected to intra-frame encoding processing, and the like. Also, the decoding unit 102
While the decoding process for the VOP data in is interrupted, the decoded image data VOPout corresponding to the VOP subjected to the decoding process immediately before the interruption of the decoding process is output from the video memory 103 as the display image data VOPdsp. . Note that the rearrangement processing of the encoded stream Vin, the decoding processing of the video stream VOPstr, and the storage of the decoded image data VOPout in the video memory are performed, respectively.
The processing is performed in the reordering unit, the decoding unit, and the video memory corresponding to each of the plurality of objects displayed simultaneously.

In the video composite unit 104,
Designated display time Td generated inside sorting unit 101
, A display screen selection signal VOPsel for selecting the decoded image data of the VOP whose VOP display time is closest to the designated display time Td is output to the video memory corresponding to each object. Then, from the video memory of each object, the decoded image data of the VOP in which the VOP display time closest to the designated display time Td is set is displayed image data V
It is supplied to the video composite unit 104 as OPdsp.

Then, the video composite unit 10
In step 4, the combining process is performed on the display image data VOPdsp read from the video memory of each object, and the combined image data Vout obtained by the combining process is output to the display unit 105. This composite image data Vout is
This is image data corresponding to one scene (one screen) of a moving image. The display unit 105 displays an image of each scene of the moving image based on the composite image data Vout. That is, in the display unit 105, the moving image is displayed by repeatedly performing the processing of displaying the image of each scene based on the designated display time Td.

Hereinafter, the rearrangement process and the decoding process for the VOP data and the process of reading the display image data from the video memory will be specifically described. FIG.
FIG. 9 is a diagram for explaining a flow of processing of an encoded stream in the video decoding device of the first embodiment. Here, VOP display times T1 to T4 (T1 <T2 <T3 <T3) are used as VOP data received by the video decoding device.
4) VOPs of VOP1 to VOP4, each of which is set
Indicates OP data. Further, in this case, the VOP data of VOP2 is received by the video decoding device with a delay that the rearrangement unit 101 cannot rearrange the VOP data of the VOP3.

In this case, the rearrangement unit 101
After the POP VOP data is output to the decoding unit 102, the VOP data of VOP2 is followed by the VOP data of VOP2.
The VOP data of VOP 3 is output to the decoding unit 102 instead of the OP data. Then, VOP of VOP3
From the VOP data of VOP2 received later than the data, the VOP of the last VOP4 received at this time
All VOP data up to the data are in the decoding unit 1
02 is output.

When the VOP data is output to the decoding unit 102 in this order, the decoding unit 102
Then, after the VOP data of VOP1 is decoded, the VOP data of VOP3 input to the decoding unit 102 before the VOP data of VOP2 is detected as defective VOP data. No decryption processing is performed. And VO
When the VOP data of VOP2 and VOP4 are input after the input of the VOP data of P3, the decoding unit 10
In 2, the decoding process is performed on the VOP data of VOP2 to VOP4. Also, the decoding unit 102
When the decoding process of the VOP data is performed in the, the decoded image data VOPou of each VOP obtained by the decoding process
t are sequentially output to the video memory 103 and stored in the memory.

As described above, the decoding unit 102
When the decoding process of the P data is performed, the decoded image data VOPout of the new VOP is not written in the video memory 103 during the interruption of the decoding process. While the decoding process is suspended, the video composite unit 10
The decoded image data VOPout of the last written VOP is output as the display image data VOPdsp based on the display screen selection signal VOPsel from 04. As a result, at the timing when the decoded images of VOP2 and VOP3 are displayed, the decoded image of VOP1 is displayed based on the display image data VOPdsp of VOP1.

In this case, when the VOP data of VOP2 is significantly delayed from the VOP data of VOP3, the decoding unit 102 waits for the reception of the VOP data of VOP2 until the VOP data of VOP2 is received. Meanwhile, when the VOP data of the intra VOP is received, the decoding process is performed from the intra VOP. For this reason, even when a stream in which VOP data of some VOPs are completely lost is input to the video decoding device 100 as the encoded stream Vin, a VOP display that is slower than the VOP in which the VOP data has disappeared is performed. When the VOP data of the intra VOP for which the time is set is input, the decoding processing of the VOP data is restarted. This is because the decoding process for the VOP data of the intra VOP is performed without referring to the image data of the decoded VOP, and the reference image data is rewritten by the decoded image data obtained by the decoding. . Therefore, a state in which the decoding process in the decoding unit 102 is stopped forever is avoided.

Next, the moving picture decoding apparatus 1 according to the first embodiment
00 will be described in detail. FIG. 3 is a block diagram for explaining the reordering unit 101 in the video decoding device according to the first embodiment. The reordering unit 101 determines, based on the input coded stream Vin, the VO
A first time extracting unit 201 for extracting a VOP display time Tin set for P (receiving VOP) and a timer (not shown) are built in, and based on reference time information in the encoded stream Vin, Designated display time Td for designating VOP display timing on the video decoding device 100 side
A display time output unit 202 for sequentially updating and outputting the specified display time Td ′, and a pre-update time output unit 203 for outputting a specified display time (display time before update) Td ′ before the update when the specified display time Td is updated. The VOP display time Tin is compared with the pre-update display time Td ′, and a control signal Csw1 corresponding to the comparison result is obtained.
And a comparison unit 204 that outputs

The reordering unit 101 selects one of the VOP display time Tin and the pre-update display time Td ′ based on the control signal Csw1, and outputs the selected time as the decoding restart time Tstart. And a buffer memory 206 for storing the encoded stream Vin.
And Here, the selection switch 205 is
The extracted VOP display time Tin is equal to the pre-update display time Td ′.
If it is earlier, select the VOP display time Tin,
If the VOP display time Tin is later than the pre-update display time Td ', the pre-update display time Td' is selected. In the buffer memory 206, the selection switch 205 sets the VOP display time Tin to the decoding restart time Tstart.
Is selected, the VOP data of the VOP whose VOP display time is set between the VOP display time Tin and the designated display time Td is output as the video stream VOPstr, while the display before updating is performed by the selection switch 205. When the time Td ′ is selected as the decoding restart time Tstart, the VOP display time outputs the VOP data of the VOP set to the time between the pre-update display time Td ′ and the designated display time Td as the video stream VOPstr. It has become.

Next, the operation of the rearrangement unit 101 will be described in detail. FIG. 4 is a diagram showing a flow of the operation of the rearrangement unit 101. First, one image sequence (object)
The encoded stream Vin corresponding to
(Step S101), the first time extracting unit 2
01, the V received as the encoded stream Vin
VOP display time corresponding to the OP data (V of the received VOP
OP display time) Tin is extracted (step S10)
2). In addition, the display time output unit 202 updates and outputs the designated display time Td indicating the display timing of the VOP on the video decoding device side based on the reference time information included in the encoded stream Vin (step). S10
3). The pre-update time output unit 202 outputs the pre-update display time Td 'in response to the update of the designated display time Td (step S104). Further, the VOP data received as the encoded stream Vin is recorded in the buffer memory 206 (Step S105).

Next, the first comparator 204 receives the received VO
The VOP display time Tin of P is compared with the display time before update Td '(step S106). As a result of this comparison, VOP
The display time Tin is earlier than the display time before update Td ′ (Tin <
Td ′), the VOP data stored in the buffer memory 206 and whose VOP display time is the time between the extracted VOP display time Tin and the designated display time Td is the video stream VOPstr. Is output (step S108). On the other hand, as a result of the comparison in step S105, when the VOP display time Tin is after the display time before update Td ′ (Tin ≧ Td ′),
VOP data stored in the buffer memory 206, the VOP display time of which is a time between the extracted pre-update display time Td ′ and the designated display time Td.
The data is output as a video stream VOPstr. That is, in this case (Tin ≧ Td ′), VOP data in the buffer memory 206 that has not been output to the decoding unit 102 and whose VOP display time is earlier than the specified display time Td is It is output as a video stream VOPstr (step S107).

As described above, in the rearrangement unit 101, the processing of steps S101 to S108 allows the early VOP
If the VOP data for which the display time is set is
When the display time is received later than the set VOP data, the VOP data reception is delayed (delay VOP data).
OP) to the VOP (display target VO) to be displayed at that time
The VOP data up to P) is newly transmitted to the decoding unit 10
2 is output. For this reason, when the VOP data of the delayed VOP is received, the decoding unit 102 sequentially performs the decoding process on the VOP data from the delayed VOP to the VOP to be displayed at this time. The image display of the display target VOP is preferably performed without deterioration in image quality.

Next, the decoding unit 102 constituting the moving picture decoding apparatus 100 according to the first embodiment will be described in detail. FIG. 5 is a block diagram for explaining the decoding unit 102 in the video decoding device according to the first embodiment. In general, the update of the designated display time Td specifying the display timing of the VOP in the video decoding device is performed at the timing at which all the VOPs corresponding to the respective VOP data received as the encoded stream are displayed. Therefore, the VOP data is transmitted from the rearrangement unit 101 to the decoding unit 102 at the update timing of the designated display time Td for each VOP.
Is long, the number of frames that can be displayed per second on the display unit 105 is V
If less than the number of OP data, the specified display time Td
, The VOP data of a plurality of VOPs may be sent from the reordering unit 101 to the decoding unit 102. However, in the decoding unit 102, VOPs of a plurality of VOPs are updated at the update timing of the designated display time Td.
The decoding process for each VOP data when data is input is performed in the same manner as the decoding process for VOP data when only VOP data of one VOP is input at the update timing of the designated display time Td. Therefore, in the following description, the update timing of the designated display time Td is
The case where the VOP data of one VOP is output from the reordering unit 101 to the decoding unit 102 will be described.

The decoding unit 102 sequentially receives the VOP data of each VOP output as the video stream VOPstr from the rearrangement unit 101, and extracts the VOP display time Tin set in the received VOP data. 2 time extracting unit 301 and the sorting unit 10
The error detection unit 302 performs an error detection process on the video stream VOPstr from 1 to check whether there is data loss due to a transmission error of the video stream VOPstr, and outputs a transmission error notification signal TRerr only when there is data loss. And

The decoding unit 102 converts the video stream VOPstr into a decoded V
A decoding process for referring to the OP image data as the reference image data VOPref is performed to output the decoded image data VOPout. In the decoding process, a grammatical error (syntax error), a semantic error (semantic error), etc. Decoder 303 that outputs decode error notification signal STerr when a stream error is detected
And an error notification unit 304 that calculates a logical sum of the transmission error notification signal TRerr and the decode error notification signal STerr and outputs an error notification signal Nerr as a logical sum signal.
And Here, the error notification signal Nerr is
This signal indicates whether at least one of the transmission error and the decoding error has occurred, or whether any error has occurred.

The decoding unit 102 receives the VOP display time Tin from the second time extraction unit 301, and receives the VOP display time T of the VOP for which the normal decoding process was performed last.
The maximum time holding unit 306 that holds “in” as the decoded maximum VOP display time Tmax, and the VOP display time Tin of the VOP (the decoding target VOP) that is the target of the decoding process,
Decrypted VOP held in maximum time holding unit 306
A second comparing unit 307 that compares the maximum display time Tmax and outputs a control signal Cpre according to the comparison result.

Here, when the decoding process for the decoding target VOP is completed correctly, the maximum time holding unit 306 sets the VOP display time Tin of the decoding target VOP to the decoded maximum time Tmax held at this time. If the time is later, the decrypted maximum time Tmax is set to the VO of the decryption target VOP.
It is updated at the P display time Tin. The control signal Cpre output from the second comparing section 307 is a decoded VOP notification signal for notifying whether or not the VOP to be decoded is a decoded VOP that has already been subjected to decoding processing. is there.

The decoding unit 102 is provided between the second time extracting unit 301 and the maximum time holding unit 306, and receives the maximum time from the second time extracting unit 301 based on the error notification signal Nerr. First open / close switch 305 for controlling supply of VOP display time Tin to section 306
Provided at a stage preceding the decoder 303, and based on the error notification signal Nerr and the control signal Cpre,
3 has a second open / close switch 308 for controlling the supply of the video stream VOPstr to the video stream VOPstr.

Here, the first open / close switch 305 is turned off when at least one of the transmission error and the decoding error has occurred, and turned on when neither the transmission error nor the decoding error has occurred. It is in a state. Also, the second open / close switch 308
Indicates that at least one of the transmission error and the decoding error has occurred, or the VO of the decoding target VOP
When the P display time Tin is a time before the decoded maximum time Tmax held in the maximum time holding unit 306 (Ti
(n ≦ Tmax), the state becomes non-conductive, on the other hand, neither a transmission error nor a decoding error has occurred, and the VOP display time Tin of the decoding target VOP is held in the maximum time holding unit 306. When the time is later than the decrypted maximum time Tmax (Tin> Tmax), the conductive state is established.

Next, the operation of the decoding unit 102 will be described in detail. FIG. 6 is a diagram showing a flow of the operation of the decoding unit 102. First, as a video stream VOPstr corresponding to one image sequence, VOP data of a predetermined VOP constituting the image sequence is received by the decoding unit 102 (step S201). Then, the second time extracting unit 301 extracts the VOP display time Tin of the received VOP data (the VOP data of the VOP to be decoded) (step S202).

The error detecting section 302 performs a transmission error detecting process for checking whether or not data is lost due to a transmission error based on the received data (video stream).
A stream error (decode error) detection process is performed based on str (step S203).

If an error is detected as a result of the error detection processing in step S203, an error notification signal Nerr from the error notification unit 304 to which the transmission error notification signal TRerr and the decoding error notification signal STerr are input is used. The second open / close switch 308 becomes non-conductive,
The supply of the video stream VOPstr to the decoder 303 is stopped. As a result, the decoding process for the video stream VOPstr in the decoder 303 is interrupted.
At this time, the first open / close switch 305 is turned off by the error notification signal Nerr from the error notification unit 304, and the VOP display time Tin of the VOP to be decoded is not supplied to the maximum time holding unit 306 (step 20).
4).

On the other hand, if no error is detected as a result of the error detection processing in step S203, the first open / close switch 305 is turned on by the error notification signal Nerr, and the second comparator 307 , Decryption target V
The VOP display time Tin of the OP is compared with the decrypted maximum time Tmax stored in the maximum time storage unit 306 (step S205). As a result of this comparison, the VOP display time T
in is a time before the maximum decrypted time Tmax (Tin ≦
Tmax), the decoding target V from the second comparing unit 307
Control signal Cpr indicating that OP is a decoded VOP
By e, the second open / close switch 308 is turned off. In the decoder 303, the video stream VOPstr
Of the video stream VOPstr is stopped (step S204).

On the other hand, as a result of the comparison in step S205,
When the VOP display time Tin is later than the decrypted maximum time Tmax (Tin> Tmax), the maximum time holding unit 30
6, the maximum decrypted time Tmax is equal to the VO of the VOP to be decrypted.
It is updated to the P display time Tin (step S206). Also, at this time, the decoding target VO from the second comparing unit 307 is output.
A control signal Cpre indicating that P is not a decoded VOP
As a result, the second open / close switch 308 becomes conductive.
Then, the decoder 303 performs a decoding process on the decoding target VOP with reference to the reference image data VOPref as necessary (step S207). And
The decoded image data VOPout for the decoding target VOP is output from the decoder 303 (step S20).
8).

Next, effects of the first embodiment will be described. In the reordering unit 101 constituting the moving picture decoding apparatus 100 according to the first embodiment, the buffer memory 206 for holding the input VOP data and the VOP display time Tin of the input VOP data are stored in the buffer memory 206.
And a comparing unit 204 for comparing the VOP display time Td ′ of the VOP data output to the decoding unit 102 with the last VOP display time.
When the VOP data of the (preceding VOP) is received later than the VOP data of the VOP (succeeding VOP) for which the late VOP display time is set, from the preceding VOP to the VOP to be displayed at that time (display target VOP) Is output to the decoding unit 102 as the video stream VOPstr, so that the decoding unit
When the P VOP data is input later than the VOP data of the succeeding VOP, the decoding process can be performed from the VOP data of the preceding VOP.

The decoding unit 102 constituting the moving picture decoding apparatus 100 according to the first embodiment performs an error detecting section 302 for detecting a transmission error, a decoding process for a video stream VOPstr, and a grammar error. And the like, and a comparison unit 307 that compares the VOP display time Tin of the input VOP data with the VOP display time Tmax of the last decoded VOP data to prevent damage due to the error. Receiving VOP data (error VOP data) or V
The decoding process for VOP data (delayed VOP data) whose OP display time is earlier than that of the latest decoded VOP data is interrupted, and when normal VOP data is input, the decoding process for VOP data is resumed. Therefore, it is possible to avoid the occurrence of remarkable deterioration in image quality, and to prevent the state in which the image quality of the display image is deteriorated from continuing.

That is, as shown in FIG.
If the OP data is received later than the VOP data of VOP3, the defective video stream VOPst
r, that is, when the decoding processing of the video stream VOPstr3 is continued by the decoding unit 102,
At the time of the decoding process of OPstr3, the decoded image data (display image data VOPdsp2) of the decoded VOP2 cannot be referred to, so that the image quality of the display image of VOP3 deteriorates. Further, in this case, even if the decoding process is restarted from the VOP data (video stream VOPstr2) of VOP2 received late, the reference image data at the restart time is updated to the decoded image data of VOP3 (display image data VOPdsp3). Therefore, it is impossible to perform decoding correctly.

Therefore, the decoding unit 102 detects the video stream VOPstr3 which has a high possibility of causing a large propagation of image degradation due to the absence of a reference image during the decoding process, and interrupts the decoding process. , And then V without causing the propagation of image quality degradation
OP data (here, video stream VOPstr
Until 2) is received, the decoded image data VOPout corresponding to the VOP1 that has been correctly decoded last is output from the bit memory 103 to the display unit 105 as the display image data VOPdsp1. Thereby, it is possible to prevent the occurrence of remarkable image quality deterioration and the propagation of the image quality deterioration.

In the first embodiment, a moving picture decoding apparatus corresponding to MPEG-4 which handles coded data of a moving picture constituted by a plurality of objects (image sequences), that is, Although the unit 101 and the decoding unit 102 process coded streams of a plurality of objects constituting one moving image, the moving image decoding apparatus performs processing on a moving image composed of one object (image sequence). May correspond to MPEG-2 or the like which handles the encoded data of the above, that is, may include only the rearrangement unit 101, the decoding unit 102, and the bit memory 103 corresponding to one image sequence.

In the first embodiment, the order of the VOPs set on the transmitting side is based on the VOP display time set in the VOP. P-VOP after encoding processing
In addition, B-VO that has been subjected to bidirectional predictive encoding
If P is included, the order in which VOPs should be displayed is VOP
Does not match the decoding order of the VOPs, and the decoding order of the VOPs cannot be directly determined from the VOP display time.
However, even when a B-VOP is included in the image sequence, a time (VOP decoding time) at which decoding processing of each VOP should be performed is obtained from the VOP display time, and the VOP display time is set to V.
By replacing with the OP decoding time, the processing in the reordering unit 101 and the decoding unit 102 according to the first embodiment can be performed in the same manner as in the case where the image sequence does not include a B-VOP.

Further, in the first embodiment, the decoder 303 decodes VOP data for which there is no corresponding reference image data (image data of a decoded VOP referred to in the inter-picture prediction decoding process). However, the decoder 303 may perform the decoding process of VOP data for which there is no corresponding reference image data.

More specifically, as shown in FIG.
The VOP data at (VOP display time T2) is VOP3 (VOP
If the VOP data is received later than the VOP data at the OP display time T3), the VOP data of the VOP3 is
When the video stream VOPstr3 is output to the decoding unit 102 as OPstr3, the video stream VOPstr3 is decoded by the decoder 303, and the decoded image data VOPout of VOP3 is stored in the video memory 103. Then, the decoded image data VOPout of VOP3 is read from the video memory 103 to the video composite unit 104 as the display image data VOPdsp3,
3 will be displayed.

In the moving picture decoding apparatus having such a configuration,
When the video stream VOPstr3 is an inter VOP, the reference image data in the video memory is
3 is updated to the decoded image data VOPout of VOP3, and the image quality deterioration in the decoded image of VOP3 is
P, and the encoded stream Vin input to the video decoding apparatus is an intra VO
In the case where only the VOP data corresponding to P is included, a normal decoded image can be displayed before the VOP data of the delayed VOP2 is input.

Further, the coded stream Vin input to the moving picture decoding device is a VOP corresponding to the inter VOP.
When the image sequence (object) does not have a shape such as a foreground but has no shape such as a background, the image sequence (object) is delayed V
Until VOP data of OP2 is input, rather than displaying a decoded image of VOP1 that has been normally decoded,
In some cases, the display image may be less unnatural if the VOP data of VOP3 input after the VOP data of VOP1 is decoded and the decoded image of VOP3 is displayed.

Further, as described above, the decoder 303 is
In a video decoding device configured to perform decoding processing on inter-VOP VOP data for which no corresponding reference image data exists, the VOP of a delayed VOP is
When the OP data arrives, the last arrived intra-V
By performing the decoding process on the VOP data from the OP to the VOP to be displayed at that time, the deterioration in the image quality of the decoded image of the inter VOP in which the corresponding reference image data does not exist is caused by the decoded image of the subsequent VOP. Can be avoided.

FIG. 8 is a block diagram for explaining a moving picture decoding apparatus having such a configuration. This video decoding apparatus performs decoding processing on inter-VOP VOP data for which there is no corresponding reference image data. The decoding process is performed on VOP data from the VOP to the VOP to be displayed at that time.

That is, the reordering unit 101a constituting the moving picture decoding apparatus extracts the VOP display time of the intra VOP based on the coded stream Vin in the reordering unit 101 of the first embodiment, and And an intra-VOP time extracting unit 201a that outputs a VOP display time Tivop (Tivop ≦ Tin) of the newest intra-VOP, which is a time before the VOP display time Tin of the first embodiment. Comparison section 20
4 based on the control signal Csw1 from the VOP display time Tivop of the most recently input intra VOP and the designated display time Td from the display time output unit 202. .

That is, when the VOP display time Tin of the input VOP data is earlier than the display time before update Td ′, the selection switch 205 sets the VOP display time T of the received VOP.
The VOP display time Tivop (Tivop ≦ Tin) of the recently input intra VOP, which is the time before in, is selected, and the VO
If the P display time Tin is not earlier than the pre-update display time Td ', the pre-update display time Td' is selected. Other configurations of the reordering unit 101a of the moving picture decoding apparatus are the same as those of the reordering unit 101 of the first embodiment.

Next, the operation of the moving picture decoding apparatus shown in FIG. 8 will be briefly described. FIG. 9 is a diagram showing an operation flow of the reordering unit 101a of the video decoding device.

In steps S101 to S106, exactly the same processing as in the rearrangement unit 101 of the first embodiment is performed. If the result of determination in step S106 is that the VOP display time Tin of the input VOP data is earlier than the pre-update display time Td ', the selection switch 20
In 5, the VOP display time Tivop of the recently input intra VOP is selected, and the selected VOP display time Tivop is selected.
Is output to the memory 206 as the decoding restart time Tstart (step S108a). At this time, VOP data from the recently input intra VOP to the VOP to be displayed at that time is output from the memory 206 to the decoding unit as a video stream VOPstr.

On the other hand, as a result of the determination in step S106,
When the VOP display time Tin is not earlier than the pre-update display time Td ′, the selection switch 205 sets the pre-update display time T
d ′ is selected, and the selected pre-update display time Td ′ is output to the memory 206 as the decoding restart time Tstart (step S107a). At this time, from memory 206, VOP display time is later than display time Td 'before update.
VOP data from the OP (that is, the VOP next to the VOP whose VOP data has already been output to the decoding unit) to the VOP to be displayed at that time is output to the decoding unit as a video stream VOPstr.

(Embodiment 2) FIG. 10 is a block diagram for explaining a moving picture decoding apparatus according to Embodiment 2 of the present invention, and shows a permutation unit constituting the moving picture decoding apparatus. I have. As described above, one moving image (video corresponding to one movie, one program, or the like) is composed of a plurality of image sequences (objects). The replacement unit is provided with at least the maximum number of image sequences (objects) displayed simultaneously when one moving image is reproduced. However, since the processing in each reordering unit is exactly the same, here, for convenience of explanation, one of a plurality of reordering units in the video decoding device is described.
(The rearrangement unit 101b) will be described.

Reordering unit 101b of the second embodiment
The first time extracting unit 201, the display time output unit 202, the pre-update time output unit 203, the first comparing unit 204, and the selection switch 20 are similar to the sorting unit 101 according to the first embodiment.
5 and a buffer memory 206. The reordering unit 101b further includes a valid time extracting unit 401 that extracts a valid time Tval of a corresponding object based on scene description information included in the encoded stream Vin input to the video decoding device; Display time output unit 2
The second comparison unit 402 compares the designated display time Td from 02 with the valid time Tval of the corresponding object, and is provided in the preceding stage of the buffer memory 206, and corresponds to the comparison result in the third comparison unit 402. An open / close switch 403 for controlling the supply of the input coded stream to the buffer memory 206 based on the control signal Csw3.

Here, the scene description information includes information indicating an arrangement of an object in each scene of a moving image, information indicating an object appearing in each scene, and the like.
From this scene description information, it is possible to obtain the timing at which each object included in the moving image appears in the display area when the moving image is reproduced. The valid time Tval of the corresponding object is the latest VOP display time corresponding to the image sequence (object) whose encoded stream has already been input to the rearrangement unit. Also, the open / close switch 40
3 is turned on when the designated display time Td is a time before the valid time Tval by the control signal Csw3 from the third comparing unit 302, and the designated display time Td is changed to the valid time Td.
When the time is later than val, it becomes non-conductive. That is, the received VOP data has the designated display time Td at the time of reception, and the VOP of the last VOP in the image sequence.
When the time is before the OP display time (valid time) Tval, the data is output to the buffer memory 206 through the open / close switch 403 as the memory input data MEMin.
On the other hand, when the designated display time Td at the time of reception is later than the VOP display time (valid time) Tval of the last VOP in the image sequence, the open / close switch 403 discards the received VOP data. , Is not output to the buffer memory 206 as the memory input data MEMin.
In FIG. 10, the same reference numerals as those in FIG. 3 indicate the same components as those of the rearrangement unit 101 of the first embodiment.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the second embodiment
Except for the operation b, the operation is the same as that of the video decoding apparatus of the first embodiment.
The operation of b will be described.

FIG. 11 is a diagram showing a flow of the operation of the rearrangement unit 101b. First, the coded stream Vin corresponding to one image sequence (object) is sorted by the reordering unit 1.
01b (step S101), and the first time extracting unit 201 outputs a VOP display time (reception VO) corresponding to the VOP data received as the encoded stream Vin.
The POP display time) Tin is extracted (step S1).
02).

The display time output unit 202 updates and outputs the designated display time Td indicating the display timing of the VOP on the video decoding device side based on the reference time information included in the encoded stream Vin. (Step S1
03). In addition, in response to the update of the designated display time Td, the pre-update time output unit 202 outputs the display time Td ′ before the update.
Is output (step S104).

Next, in the third comparing section 402, the designated display time Td from the display time output section 202 is compared with the valid time Tval output from the valid time extracting section 401a (step S301). As a result of the comparison, when the designated display time Td is a time after the valid time Tval, the third
The open / close switch 403 is turned off by the control signal Csw3 from the comparison unit 402, and the received VOP data is discarded (step S302). On the other hand, as a result of the comparison in step S301, when the designated display time Td is a time before the valid time Tval, the open / close switch 403 is turned on by the control signal Csw3 from the third comparing unit 402, and is received. The VOP data is recorded in the buffer memory 206 as the memory input data MEMin (step S105).

Thereafter, the rearranging unit 1 of the first embodiment
01, the VOP display time T in the first comparison unit 204
in and the display time before update Td ′ (step S1)
06), and VOP data output processing according to the comparison result (steps S107 and S108). That is,
The VOP display time Tin of the received VOP is the pre-update display time T
If it is earlier than d ′ (Tin <Td ′), the VOP display time is the VOP display time Tin of the reception VOP and the designated display time T
The VOP data of the VOP which is the time between d is output from the buffer memory 206 (step S108). On the other hand, as a result of the comparison in step S105, the VOP display time Tin is later than the pre-update display time Td ′ (Tin ≧ T
d '), the VOP display time is the display time before update Td'
The VOP data of the VOP which is a time between the specified time and the designated display time Td is output from the buffer memory 206. That is, in this case (Tin ≧ Td ′), the buffer memory 2
06, the VOP data not yet output to the decoding unit 102, and the VOP display time is
VOP data earlier than the designated display time Td is output as the video stream VOPstr (step S10).
6).

Next, the operation and effect of the permutation unit corresponding to each object in the moving picture decoding apparatus according to the second embodiment will be described. In the moving picture decoding apparatus, the rearrangement processing and the decoding processing of the VOP data of the simultaneously displayed objects are performed in parallel by the rearrangement unit and the decoding unit corresponding to each of the simultaneously displayed objects. However, the number of objects displayed simultaneously differs depending on the scene of the moving image.

More specifically, even when one moving image includes six image sequences (that is, objects A to F), when one moving image is reproduced, all objects are moved from the first scene. The last scene is not displayed. For example, as shown in FIG. 12A, three objects A, B, and C are generated between times t0 and t1, and three objects A and B are output between times t1 and t2.
E and F are two objects A and D between time t2 and t3,
When displayed as a composite image, the maximum number of image sequences (objects) displayed simultaneously is three.

In such a moving picture decoding apparatus for decoding and reproducing a coded stream of a moving picture, at least 3
One rearrangement unit (first to third rearrangement units) is required. That is, in the period from time t0 to t1, the encoded streams of the three objects A, B, and C are subjected to the reordering processing by the first to third reordering units, respectively. Time t1 to t
In the period 2, the encoded streams of the three objects A, E, and F are subjected to rearrangement processing by the first to third rearrangement units, respectively. Further, during the period from time t2 to t3, the two objects A and D are subjected to the sorting process by the first and second sorting units, respectively. FIG. 12B shows which object's encoded stream is processed by the reordering unit (decoding unit) in each of the above periods.

At this time, in the first rearrangement unit, the coded stream of the object A is rearranged from the start of reproduction of the moving image (time t0) to the end of reproduction (time t3). In the second rearrangement unit, the rearrangement process of the coded stream of the object B is performed between times t0 and t1, the rearrangement process of the coded stream of the object E is performed between times t1 and t2, and between the times t2 and t3. Performs a process of rearranging the encoded stream of the object D.
In the third rearrangement unit, the object C is used between the times t0 and t1.
Of the coded stream of the object F during the time t1 to t2,
Between 2 and t3, the coded stream rearrangement process is not performed.

Therefore, in a moving picture decoding apparatus in which the maximum number of image sequences (objects) displayed simultaneously (maximum number of image sequences) is set to 3, the rearrangement unit, the decoding unit, and the video memory are 3 Time t0 at which the rearrangement process and the decoding process are performed on the encoded streams for the three objects, for example, as described above.
In the period from t1 to t1 or the period from time t1 to t2, there is no room for the number of objects that can be processed.

However, the VOP data of the VOP (preceding VOP) for which the early VOP display time is set is changed to the late VOP data.
When the OP display time is input to the video decoding device later than the set VOP (subsequent VOP), the buffer memory 206 displays the display indicated by the designated display time Td at the input time from the preceding VOP. VOP data up to the power VOP (display target VOP) is output to the decoding unit as a video stream VOPstr.

In this case, the number of image sequences to be simultaneously decoded may exceed the maximum number of image sequences in the moving picture decoding apparatus. Specifically, the time t1 to t2 when the VOP data of the object E is input to the second sorting unit
When the VOP data (the VOP data of the delayed VOP) arriving at the delay of the object B is input to the second rearrangement unit during the period of, the second rearrangement unit outputs the last of the object B from the delay VOP. The VOP data stored in the buffer memory up to the VOP is output to the second decoding unit.

In this case, there is no problem in a moving picture decoding apparatus having a maximum number of image sequences, that is, a number of image sequences that can be simultaneously decoded, but in a moving image decoding apparatus having no maximum number of image sequences, There is a possibility that the number of image sequences that need to be simultaneously decoded exceeds the maximum number of image sequences. That is, in the moving picture decoding apparatus in which the maximum number of image sequences is three, the first, second, and
In the third decoding unit, objects A, E,
Since the decoding process is performed on the object F, the number of image sequences that need to be simultaneously decoded when the VOPVOP data of the object B is output from the second rearrangement unit to the second decoding unit Becomes 4, which is 3 which is the maximum number of image sequences that can be simultaneously decoded in the video decoding device.
Will be exceeded.

Therefore, the rearranging unit 1 of the second embodiment is
02, the last VOP of each image sequence (object) based on the scene description information included in the input encoded stream
Is extracted, and the designated display time Td at the time when the VOP data of the image sequence corresponding to the rearrangement unit 102 is received is later than the valid time Tval of the corresponding image sequence , The input V
OP data is discarded.

As described above, in the reordering unit 101b of the second embodiment, the VOP data of the preceding VOP is replaced with the VO of the succeeding VOP, similarly to the reordering unit 101 of the first embodiment.
When the data is received later than the P data, the VOP data from the preceding VOP to the display target VOP to be displayed at that time is output to the decoding unit 102 as the video stream VOPstr.
When the P VOP data is input later than the VOP data of the succeeding VOP, the decoding process can be restarted from the VOP data of the preceding VOP.

The rearranging unit 1 of the second embodiment
01b, a valid time extracting unit 40 that extracts the VOP display time of the last VOP of the object as a valid time Tval based on the scene description information included in the encoded stream Vin.
1 and the designated display time Td at the time when the VOP data is received is set to the effective time Tva of the object corresponding to the VOP data.
When the designated display time Td at the time of receiving the VOP data has passed the valid time Tval, the VOP data of the received VOP is discarded, and thus a decoding process is performed. VOPs that are no longer needed
Data can be prevented from being input to the buffer memory 206.

As a result, it is possible to perform a decoding process in consideration of the number of image sequences that can be simultaneously decoded by the video decoding device. In other words, in a moving picture decoding apparatus that does not have enough room for the maximum number of image sequences (the number of image sequences that can be simultaneously decoded), the number of image sequences that need to be decoded simultaneously exceeds the maximum number of image sequences. Can be avoided. Further, since the VOP data of the VOP for which it is no longer necessary to perform the decoding process is not output to the buffer memory 206, the recording area of the buffer memory 206 can be effectively used.

In the second embodiment, the designated display time Td generated at a predetermined timing on the video decoding device side is the valid time (ie, the last VOP display time of the object corresponding to the reception VOP). Tval, if later
Although the VOP data of the received VOP is always discarded, even if the designated display time Td is later than the valid time Tval, there is a margin in the number of image sequences to be simultaneously decoded by the video decoding device. Sometimes, V of the received VOP
The OP data may be supplied to the buffer memory 206.

For example, at times t2 to t3 shown in FIG.
When the number of image sequences “2” to be simultaneously decoded is smaller than the maximum number of image sequences “3” of the video decoding device as in the period of, the delayed VOP data of the object E is in the second average. When input to the replacement unit, the delay V
V from OP to display target VOP to be displayed at that time
This can be handled by outputting the OP data to a third decoding unit that has not performed the decoding process at that time. In this case, the second decoding unit corresponding to the second rearrangement unit can continue the decoding processing of the VOP data for the object D.

(Embodiment 3) FIG. 13 is a block diagram for explaining a moving picture decoding apparatus according to Embodiment 3 of the present invention, and shows a permutation unit constituting the moving picture decoding apparatus. I have. The reordering unit 101c in the moving picture decoding apparatus according to the third embodiment includes a first time extracting unit 201, a display time output unit 202, and a pre-update time output unit 203, similarly to the reordering unit 101 according to the first embodiment. , A first comparison unit 204, and a buffer memory 206.
The reordering unit 101c includes an oldest time extracting unit 501 that extracts the VOP display time of the oldest VOP among the VOPs whose VOP data is recorded in the buffer memory 206 as the oldest time Tmin, VOP display time T of the received VOP output from the time extraction unit 201
in is compared with the oldest time Tmin, and a fourth comparison unit 502 that outputs a control signal STRsw according to the comparison result is provided at a stage preceding the buffer memory 206,
02 based on the control signal STRsw corresponding to the comparison result in the buffer memory 20 of the input coded stream.
And an open / close switch 503 for controlling the supply to the power supply 6. Also, the rearranging unit 101c is the same as that of the first embodiment.
Instead of the selection switch 205 of the rearrangement unit 101,
Based on the control signal Csw1 from the first comparing section 204 and the control signal STRsw from the fourth comparing section 502, the VOP display time Tin and the display time output section 2 of the received VOP are displayed.
A selection switch 205 c for selecting one of the designated display times Td from 02 and outputting it to the buffer memory 206 is provided.

Here, the open / close switch 503 is connected to the fourth
Of the received V
When the VOP display time Tin of the OP is earlier than the oldest time Tmim, the state becomes non-conductive, and when the VOP display time Tin of the received VOP is the time after the oldest time Tmim, the state becomes conductive. That is, in the received VOP data, the VOP display time Tin is stored in the buffer memory 20.
6, when the time is earlier than the VOP display time (oldest time) Tmim of the oldest VOP among the VOPs whose VOP data is recorded, the data is discarded by the open / close switch 503 and stored in the buffer memory as the memory input data MEMin. 2
Not output to 06. On the other hand, the received VOP data is output to the buffer memory 206 as the memory input data MEMin via the open / close switch 503 when the VOP display time Tin is a time after the oldest time Tmim.

In the selection switch 205c, V
If the OP display time Tin is earlier than the earliest time Tmim, the control signal STRsw from the fourth comparing unit 502 regardless of the control signal Csw1 from the first comparing unit 204.
, The pre-update display time Td 'is selected, and the pre-update display time Td' is output to the buffer memory 206 as the decoding restart time Tstart. On the other hand, if the extracted VOP display time Tin is a time after the earliest time Tmin, the selection switch 205c sets the VOP display time Tin before the update in accordance with the control signal Csw1 of the first comparison unit 204. One of the display times Td 'is selected, and the selected time is output to the buffer memory 206 as the decoding restart time Tstart. That is, in this case, the selection switch 205c
When the VOP display time Tin is a time after the pre-update display time Td ', the pre-update display time Td' is selected.
When the P display time Tin is earlier than the pre-update display time Td ', the VOP display time Tin is selected. In FIG. 13, the same reference numerals as those in FIG. 3 indicate the same components as those of the rearrangement unit 101 of the first embodiment.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the third embodiment is performed by
Except for the operation c, the operation is the same as that of the video decoding apparatus of the first embodiment.
The operation of c will be described.

FIG. 14 is a diagram showing a flow of the operation of the rearrangement unit 101c. First, similarly to the reordering unit 101 of the first embodiment, reception of VOP data input as an encoded stream Vin (step S101) and extraction of a VOP display time Tin of the received VOP (step S10)
2), output of the designated display time Td (step S103),
The display time before update Td ′ is output (step S104).

Thereafter, the oldest time extracting section 501 of the rearranging unit 101c stores the VO in the buffer memory 206 based on the time information Itime from the buffer memory 206.
The oldest VO among VOPs in which P data is recorded
The VOP display time of P (oldest VOP) is extracted as the oldest time Tmim (step S400).

Subsequently, in the fourth comparing section 502, the VOP display time Tin output from the first time extracting section 201 is compared with the oldest time Tmi output from the oldest time extracting section 501.
is compared with n (step S401). As a result of this comparison, if the VOP display time Tin is a time before the oldest time Tmin, the control signal ST from the fourth comparing unit 502
By Rsw, the open / close switch 503 is turned off, and the VOP data of the received VOP is discarded (step S402). At this time, even if the control signal Csw1 from the first comparing unit 204 indicates that the VOP display time Tin is earlier than the display time before update Td ′, the fourth step in the selection step 205c is Comparison section 50
2 by the control signal STRsw from the display time T before the update.
d 'is selected.

On the other hand, as a result of the comparison in step S401, if the VOP display time Tin is a time after the oldest time Tmin, the VOP recorded in the buffer memory 206
Of the OP data, the VOP display time is the V of the received VOP.
The VOP data which is a time within the range from the OP display time Tin to the designated display time Td is newly decoded by the decoding unit 102.
Can be retransmitted. Therefore, the processes of steps S105 to S108 are performed as in the first embodiment.

That is, the open / close switch 503 is turned on by the control signal STRsw from the fourth comparing section 502, and the VOP data of the received VOP is stored in the memory input data ME.
Min is supplied to the buffer memory 206 and recorded in the buffer memory 206 (step S105).

Thereafter, the rearranging unit 1 of the first embodiment
01, the VOP display time T in the first comparison unit 204
in and the display time before update Td ′ (step S1)
06), and VOP data output processing according to the comparison result (steps S107 and S108). That is,
If the data VOP display time Tin is earlier than the pre-update display time Td ′ (Tin <Td ′), the VOP display time becomes
The VOP data of the VOP which is a time between the VOP display time Tin of the OP and the designated display time Td is stored in the buffer memory 20.
6 (Step S108). On the other hand, as a result of the comparison in step S105, when the VOP display time Tin is after the pre-update display time Td ′ (Tin ≧ Td ′), the VOP display time is set between the pre-update display time Td ′ and the designated display time Td. The VOP data of the VOP that is the time between
Output from the buffer memory 206 (step S10
6).

Next, the function and effect of the third embodiment will be described. As described in the first embodiment, in the rearrangement unit 101 of the first embodiment, the display time (designated display time) Td specified by the moving picture decoding device due to the transmission path delay of the VOP data. Even decoding unit 102
If there is VOP data that cannot be output to the VOP, all the VOP data from the delayed VOP received late with the VOP data to the display target VOP to be displayed at that time are
It is sent to the decoding unit 102 again. As a result, even if some VOP data is received with a delay,
In the decoding unit 102, the decoding process is performed correctly,
The image quality deterioration due to the delayed VOP is caused by the VOP following the delayed VOP.
Of the image quality, which adversely affects the image quality of the image, is avoided.
However, although VOP data is temporarily stored in the buffer memory 206, due to limitations on memory capacity and the like,
In the buffer memory 206, VOP data is sequentially deleted from the earliest reception time.

Therefore, the VOP display time Tin of the delayed VOP
Is earlier than the oldest time Tmin, the delay V
Since a part of the VOP data from the OP to the display target VOP has disappeared from the buffer memory 206, the VOP data from the delay VOP to the display target VOP recorded in the buffer memory 206 is newly decoded.
2, the decoding unit 102 outputs the delay V
It is impossible to perform a correct decoding process for each VOP from the OP to the display target VOP. Further, the reference image data referred to in the decoding process of the inter VOP is a decoding error VOP that has not been correctly decoded in the decoding process for the next VOP in the decoding unit 102. It is rewritten with the decoded image data. As a result, sending the VOP data from the delayed VOP to the display target VOP from the rearrangement unit to the decoding unit 102 again causes the propagation of image quality deterioration.

Therefore, in the second embodiment, delay VO
Even if the VOP display time Tin of P is earlier than the display time Td ′ before the update, the VOP display time Tin is a time after the oldest time Tmin, that is, the buffer memory 2
06, the VOP data from the delayed VOP to the display target VOP is recorded only when VOP data of all the VOPs whose VOP display time is in the range from the VOP display time Tin of the delayed VOP to the designated display time Td is recorded. The VOP data is converted from the reordering unit 101c to the decoding unit 10 again.
I send it to 2.

In other words, the earliest time extracting unit 501
Each VO whose data is stored in the buffer memory 206
The VOP display time of P is checked, and the oldest time Tmin is output to the fourth comparing unit 502. The fourth comparison unit 502
The VOP display time Tin of the OP is compared with the oldest time Tmin,
The control signal STRsw according to the comparison result is transmitted to the open / close switch 50.
3 and the selection switch 205c. If the VOP display time Tin is a time after the oldest time Tmin, the switch 5
03 is in the conductive state, and the VOP data of the received VOP is recorded in the buffer memory 206. On the other hand, if the VOP display time Tin is earlier than the oldest time Tmin, the switch 503 is turned off, and the VOP data of the received VOP is discarded. The selection switch 205c sets the VOP display time T based on the comparison result of the first comparison unit 204.
If in is a time before the display time before update Td ′, V
The OP display time Tin is selected as the decoding restart time Tstart, while the VOP display time Tin is set to the pre-update display time Td ′.
If it is a subsequent time, the pre-update display time Td ′ is selected as the decoding restart time Tstart. However, the VO of the received VOP
If the P display time Tin is earlier than the oldest time Tmin, the selection switch 205c changes the display time before update Td 'to the decoding restart time Tstar regardless of the result of comparison by the first comparing unit 204.
Select as t.

As described above, in the rearrangement unit 101c according to the third embodiment, as in the rearrangement unit 101 according to the first embodiment, the VOP data of the VOP (preceding VOP) for which the early VOP display time is set is later. When the VOP display time is received later than the VOP data of the set VOP (subsequent VOP), the VOP data from the preceding VOP to the display target VOP to be displayed at that time is used as the video stream VOPstr as the decoding unit 102. Therefore, when the VOP data of the preceding VOP is input later than the VOP data of the succeeding VOP, the decoding unit can restart the decoding process from the VOP data of the preceding VOP.

The rearranging unit 1 of the third embodiment
01c, the earliest time extracting unit that extracts the oldest time (oldest time) Tmin from the VOP display times of the VOPs whose VOP data is stored in the buffer memory 206 based on the time information Itime from the buffer memory 206. 50
1 only when the VOP display time is within the range from the VOP display time Tin of the delayed VOP to the designated display time Td, and the VOP data of all the VOPs is recorded in the buffer memory 206. Display target V
The VOP data up to the OP is decoded again by the decoding unit 102
Therefore, if the delayed VOP is older than the oldest VOP in which the VOP data is stored in the buffer memory 206, the deterioration of the image quality is not propagated in the decoding process of the VOP data from the delayed VOP to the display target VOP. This can be avoided.

(Embodiment 4) FIG. 15 is a block diagram for explaining a moving picture decoding apparatus according to Embodiment 4 of the present invention, and shows a reordering unit constituting the moving picture decoding apparatus. I have. The reordering unit 101d in the moving picture decoding apparatus according to the fourth embodiment includes a first time extracting unit 201, a valid time extracting unit 401, a display time outputting unit 202, similar to the reordering unit 101b according to the second embodiment. Pre-update time output section 203, first comparing section 204, third comparing section 4
02 and a buffer memory 206. The reordering unit 101d includes an oldest time extracting unit 501 that extracts the VOP display time of the oldest VOP among the VOPs whose VOP data is recorded in the buffer memory 206 as the oldest time Tmin, Time extraction unit 201
Is compared with the oldest time Tmin, and the control signal S corresponding to the comparison result is output.
A fourth comparison unit 502 that outputs TRsw.

The rearranging unit 101d is the same as the open / close switch 403 of the rearranging unit 101b of the second embodiment.
Instead of the control signal Csw from the third comparing unit 402
An open / close switch 403d for controlling the supply of the input coded stream to the buffer memory 206 based on the control signal STRsw from the third comparison unit 502 and the control signal STRsw from the fourth comparison unit 502 is provided. Further, the rearrangement unit 101d is different from the rearrangement unit 101b according to the second embodiment in that the selection switch 205 is replaced with a control signal Csw1 from the first comparison unit 204.
Based on the control signal STRsw from the fourth comparing unit 502 and one of the VOP display time Tin of the reception VOP and the designated display time Td from the display time output unit 202, the selection to output to the buffer memory 206 Switch 205d
It has.

Here, the open / close switch 403d is
The designated display time Td at the time when the OP data is received is a time before the valid time Tval (Td ≦ Tval), and the VOP display time Tin of the received VOP is a time after the oldest time Tmim (Tin ≧ Tmim). On the other hand, when the specified display time Td at the time of receiving the VOP data is a time after the valid time Tval (Td> Tval),
Alternatively, the VOP display time Tin of the reception VOP is the oldest time T
When the time is before the mim (Tin <Tmim), the state becomes non-conductive.

That is, the received VOP data has the specified display time Td at the time of the reception and the last VO of the image sequence.
The VOP display time (valid time) of P is a time before the valid time Tval, and the VOP display time Tin of the received VOP is the oldest VOP display time (VOP display time) of the VOPs whose VOP data is recorded in the buffer memory 206. Old time) T
When the time is after mim, the buffer memory 2 is input as memory input data MEMin via the open / close switch 403d.
06 is output. On the other hand, the received VOP data
When the designated display time Td at the time of reception is later than the VOP display time (valid time T) val of the last VOP of the image sequence, or the VOP display time T of the received VOP
When “in” is a time before the VOP display time (oldest time) Tmim of the oldest VOP among the VOPs whose VOP data is recorded in the buffer memory 206, the open / close switch 403 d discards the “in” and the buffer memory 206
Is not output to

In the selection switch 205d, V
If the OP display time Tin is earlier than the earliest time Tmim, the control signal STRsw from the fourth comparing unit 502 regardless of the control signal Csw1 from the first comparing unit 204.
, The pre-update display time Td 'is selected, and the pre-update display time Td' is output to the buffer memory 206 as the decoding restart time Tstart. On the other hand, if the extracted VOP display time Tin is a time after the earliest time Tmin, the selection switch 205d sets the VOP display time Tin before the update in accordance with the control signal Csw1 of the first comparison unit 204. One of the display times Td 'is selected, and the selected time is output to the buffer memory 206 as the decoding restart time Tstart. That is, in this case, the selection switch 205d
When the VOP display time Tin is a time after the pre-update display time Td ', the pre-update display time Td' is selected.
When the P display time Tin is earlier than the pre-update display time Td ', the VOP display time Tin is selected. In FIG. 15, the same reference numerals as those in FIGS. 10 and 13 denote the same components as those of the rearrangement unit 101b of the second embodiment and the rearrangement unit 101c of the third embodiment.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the fourth embodiment is performed by
The operation other than the operation d is the same as the operation of the moving picture decoding apparatus according to the second embodiment.
The operation of d will be described.

FIG. 16 is a diagram showing a flow of the operation of the rearrangement unit 101d. First, similarly to the rearrangement unit 101b of the second embodiment, reception of VOP data input as an encoded stream Vin (step S101), extraction of the VOP display time Tin of the received VOP (step S10)
2), output of the designated display time Td (step S103),
The display time before update Td ′ is output (step S104).

Thereafter, the oldest time extracting unit 501 of the rearrangement unit 101d stores the VO in the buffer memory 206 based on the time information Itime from the buffer memory 206.
The oldest VO among VOPs in which P data is recorded
The VOP display time of P (oldest VOP) is extracted as the oldest time Tmim (step S400).

Next, in the third comparing section 402, the designated display time Td output from the display time output section 202 is compared with the valid time Tval output from the valid time extracting section 401a.
Is compared with (step S301). As a result of this comparison,
When the designated display time Td at the time of receiving the VOP data is later than the valid time Tval (Td> Tval), the third
The open / close switch 403d is turned off by the control signal Csw3 from the comparison unit 402, and the received VOP data is discarded (step S302). On the other hand, as a result of the comparison in step S301, when the designated display time Td is a time before the valid time Tval (Td ≦ Tval), the fourth comparing unit 502 further outputs the first time extracting unit 20
1 is compared with the oldest time Tmin output from the oldest time extracting unit 501 (step S401).

As a result of this comparison, if the VOP display time Tin is earlier than the oldest time Tmin (Tin <Tmin), the open / close switch 403d is turned off by the control signal STRsw from the fourth comparison unit 502. State and the VOP data of the received VOP is discarded (step S40).
2). At this time, when the control signal Csw1 from the first comparing unit 204 is changed to the VOP display time Tin, the pre-update display time T
Even if it indicates that the time is earlier than d ', the selection switch 205d sets the fourth comparison unit 403d
Display time Td ′ before update by the control signal STRsw from
Is selected.

On the other hand, as a result of the comparison in step S401, if the VOP display time Tin is a time after the oldest time Tmin (Tin ≧ Tmin), the buffer memory 206
, VOP data whose VOP display time is within the range from the VOP display time Tin of the received VOP to the designated display time Td can be retransmitted to the decoding unit 102 again. Therefore, the processes of steps S105 to S108 are performed as in the second embodiment. That is, the open / close switch 403d is turned on by the control signal STRsw from the fourth comparing unit 502, and the VOP data of the received VOP is stored in the memory input data ME.
Min is supplied to the buffer memory 206 and recorded in the buffer memory 206 (step S105).

Thereafter, the rearranging unit 1 of the second embodiment
01, the VOP display time T in the first comparison unit 204
in and the display time before update Td ′ (step S1)
06), and VOP data output processing according to the comparison result (steps S107 and S108). That is,
If the data VOP display time Tin is earlier than the display time before update Td ′ (Tin <Td ′), the VOP display time is changed to the reception VOP VOP display time Tin and the designated display time T.
The VOP data of the VOP which is the time between d is output from the buffer memory 206 (step S108). On the other hand, as a result of the comparison in step S105, the VOP display time Tin is changed to the time after the pre-update display time Td ′ (Tin ≧ T
d '), the VOP display time is the VOP of the VOP which is a time between the display time before update Td' and the designated display time Td.
OP data is output from the buffer memory 206 (step S106).

As described above, in rearrangement unit 101d of the fourth embodiment, similar to rearrangement unit 101 of the first embodiment, when the VOP data of the preceding VOP is received later than the VOP data of the succeeding VOP, VOP data from the VOP to the display target VOP to be displayed at that time is used as a video stream VOPstr by the decoding unit 102.
, The decoding unit outputs V
When the OP data is input later than the VOP data of the succeeding VOP, the decoding process can be restarted from the VOP data of the preceding VOP.

Also, the rearranging unit 1 of the fourth embodiment
01d, a valid time extracting unit 401 that extracts the VOP display time of the last VOP of the object as a valid time Tval based on the scene description information included in the encoded stream Vin, as in the reordering unit 101b of the second embodiment. And V
The designated display time Td at the time when the OP data is received
A third comparing unit 402 for comparing the valid time Tval of the object corresponding to the VOP data with the specified display time Td at the time of receiving the VOP data when the valid time Tval has passed; Is discarded, so that it is possible to avoid inputting the VOP data of the VOP for which the decoding process does not need to be performed to the buffer memory 206. This makes it possible to perform a decoding process in consideration of the number of image sequences that can be simultaneously decoded by the video decoding device. Further, since the VOP data of the VOP for which it is no longer necessary to perform the decoding process is not output to the buffer memory 206, the recording area of the buffer memory 206 can be effectively used.

Also, the rearranging unit 1 of the fourth embodiment
01d, the oldest time (oldest time) Tmin among the VOP display times of the VOPs whose VOP data is stored in the buffer memory 206 is set to the time from the buffer memory 206, as in the rearrangement unit 101c of the third embodiment. An oldest time extracting unit 501 for extracting based on the information Itime is provided, and the VOP display time is the VOP display time Tin of the delayed VOP.
VOP data from the delayed VOP to the display target VOP is newly sent to the decoding unit 102 only when the VOP data of all the VOPs within the range from the time to the designated display time Td is recorded in the buffer memory 206. The delay VOP is stored in the buffer memory 206.
If the data is older than the oldest stored VOP,
In the decoding process of VOP data from the delayed VOP to the display target VOP, it is possible to avoid the propagation of image quality degradation.

(Embodiment 5) FIG. 17 is a block diagram for explaining a moving picture decoding apparatus according to Embodiment 5 of the present invention, and shows a decoding unit constituting the moving picture decoding apparatus. I have. The decoding unit 102e in the video decoding apparatus according to the fifth embodiment includes a second time extracting unit 30 like the decoding unit 102 according to the first embodiment.
1, error detection unit 302, decoder 303, open / close switch 305, maximum time holding unit 306, second comparison unit 30
7 and an open / close switch 308.

Then, this decoding unit 102e
Based on the video stream VOPstr output from the rearrangement unit (see FIG. 1) of the video decoding device, it is determined whether or not this video stream VOPstr includes shape information representing the shape of an object, and the video stream VOPstr is added to the video stream VOPstr. Shape presence / absence determining unit 6 that outputs a shape notification signal Nshape indicating whether the corresponding image sequence (object) has a shape or not.
01 is provided.

The decoding unit 102e is the same as the error notifying unit 30 of the decoding unit 102 of the first embodiment.
4 based on the shape notification signal Nshape from the shape presence / absence determination unit 601, the transmission error notification signal TRerr from the error detection unit 302, and the decode error notification signal STerr from the decoder 303.
Switch control signal C for turning off 05 and 308
A switch control unit 304e that outputs off is provided.

Here, the switch control section 304e transmits the transmission error notification signal TRerr and the decoding error notification signal S
At least one of Terr indicates that an error has occurred, and the shape notification signal Nshape is the video stream VO.
Only when notifying that the image sequence (object) corresponding to Pstr has a shape, a switch control signal Coff for making the switches 305 and 308 nonconductive is output. Therefore, when no error is detected or when the video stream VOP
When the image sequence (object) corresponding to str has no shape, the open / close switches 305 and 308 are not turned off by the switch control signal Coff. However, in this case, the open / close switch 308 is connected to the second comparison unit 3
07 is controlled by a control signal Cpre from the controller 07. In FIG. 17, the same reference numerals as those in FIG. 5 indicate the same components as those of the decoding unit 102 according to the first embodiment.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the fifth embodiment is the same as that of the decoding unit 10
Except for the operation of 2e, the operation is the same as that of the video decoding device of the first embodiment.
02e will be described. FIG. 18 is a diagram showing a flow of the operation of the decoding unit 102e.

First, in the decoding unit 102e according to the fifth embodiment, as in the decoding unit 102 according to the first embodiment, the video stream VO corresponding to one image sequence is
As Pstr, V of a predetermined VOP constituting the image sequence
The OP data is received by the decoding unit 102 (step S201), and subsequently, the second time extracting unit 301
The VOP display time Tin of the received VOP data (VOP data of the VOP to be decoded) is extracted (step S2).
02).

After that, in the decoding unit 102e according to the fifth embodiment, the shape extraction unit 601 determines whether or not the image sequence (object) has a shape signal (step S501). If the result of this determination is that the image sequence has a shape, detection of a transmission error by the error detection unit 302 and detection of a stream error by the decoder 303 are performed (step S203). On the other hand, if the result of the determination in step S501 is that the image sequence does not have a shape, the error detection in step S203 is not performed, and the second comparing unit 307 determines
The VOP display time Tin of the decoding target VOP is compared with the decrypted maximum time Tmax stored in the maximum time storage unit 306 (step S205).

If an error is detected as a result of the error detection processing in step S203, the transmission error notification signal TRerr and the decoding error notification signal STerr are input by the switch control signal Coff from the switch control unit 304e. , The second open / close switch 308 becomes non-conductive, and the decoder 30 of the video stream VOPstr
The supply to 3 is stopped. As a result, the decoding process for the video stream VOPstr in the decoder 303 is interrupted. At this time, the first open / close switch 305 is turned off by the switch control signal Coff from the switch control unit 304e, and the VOP display time Tin of the decoding target VOP is not supplied to the maximum time holding unit 306 (step 204). .

On the other hand, if no error is detected as a result of the error detection processing in step S203, the first open / close switch 305 is turned on, and the second comparator 30
At 7, the VOP display time Tin of the VOP to be decoded is changed to the decoded maximum time Tm held in the maximum time holding unit 306.
ax (Step S205). As a result of this comparison, when the VOP display time Tin is a time before the maximum decrypted time Tmax (Tin ≦ Tmax), the second comparison unit 30
7, the control signal Cpre indicating that the VOP to be decoded is a decrypted VOP, the second open / close switch 3
08 is turned off. At this time, in the decoder 303, the decoding process for the video stream VOPstr is interrupted by stopping the supply of the video stream VOPstr to the decoder 303 (step S204).

On the other hand, as a result of the comparison in step S205,
When the VOP display time Tin is later than the decrypted maximum time Tmax (Tin> Tmax), the maximum time holding unit 30
6, the maximum decrypted time Tmax is equal to the VO of the VOP to be decrypted.
It is updated to the P display time Tin (step S206). Also, at this time, the decoding target VO from the second comparing unit 307 is output.
A control signal Cpre indicating that P is not a decoded VOP
As a result, the second open / close switch 308 becomes conductive.
Then, the decoder 303 performs a decoding process on the decoding target VOP with reference to the reference image data VOPref as necessary (step S207). And
The decoded image data VOPout for the decoding target VOP is output from the decoder 303 (step S20).
8).

The operation and effect of the fifth embodiment will be described below. In the encoding process for an image sequence in which the size of each image (VOP) does not change, even if the image quality is deteriorated on some screens due to the lack of some of the received data on the receiving side, the image quality deterioration is caused by another image. It has little effect on the screen. On the other hand, in the encoding process for an image sequence having a shape, a shape signal encoding process and a pixel value encoding process depending on the shape of an image that has been locally decoded are performed at the time of this encoding process. On the receiving side, the loss of a part of the received data causes the deterioration of the image quality to be propagated, and as a whole, the image quality significantly deteriorates.

In general, in applications where real-time performance is required,
Encoding processing is often used for an image sequence in which the size of an image does not change, that is, has no shape. On the other hand, in applications that do not require much real-time processing, encoding processing for an image sequence having a shape is often used. Therefore, in the decoding process of an image sequence not having a shape that requires real-time performance, it is better to continue decoding even if some image quality degradation occurs due to lack of received data or the like.

Therefore, in the fifth embodiment, when the input video stream VOPstr has shape information, when the occurrence of a transmission error or a stream error is detected, the video stream VOPstr is used.
Switch 30 so that the decoding process for
When the input video stream VOPstr has no shape information, the decoding process for the video stream VOPstr is continued even when the occurrence of a transmission error or a stream error is detected. The on / off switch 308 is kept conductive.

As described above, in the decoding unit 102e of the fifth embodiment, the input video stream VOPst
The video stream VOPstr is provided with a shape presence / absence determining unit 601 for determining whether or not the video stream VOPstr includes shape information representing the shape of an object based on r, and an error of the video stream VOPstr is detected only when the video stream VOPstr has shape information. When the video stream VOPst is detected,
Since the decoding process for r is interrupted, the adverse effect on the image quality such as a transmission error is small, and for an image sequence having no shape signal, the decoding process can be performed so as not to impair the real-time property, Moreover, for an image sequence having a shape signal, which has a large adverse effect on image quality such as a transmission error,
Deterioration of image quality due to transmission errors and the like can be reduced.

In the fifth embodiment, whether or not to interrupt the decoding process when an error is detected is determined according to whether or not the input video stream VOPstr has shape information. Alternatively, whether or not to interrupt the decoding process when an error is detected may be determined by a user operation according to the user's preference or application.

(Embodiment 6) FIG. 19 is a block diagram for explaining a video decoding apparatus according to Embodiment 6 of the present invention, and shows a decoding unit constituting the video decoding apparatus. I have. The decoding unit 102f according to the sixth embodiment generates an error processing instruction signal Ec indicating whether or not to interrupt the decoding processing when an error is detected by a user operation.
sel is generated, and it is determined whether or not to interrupt the decoding process when an error is detected, based on the error-time processing instruction signal Ecsel.

That is, the decoding unit 102f is the same as the decoding unit 102e of the fifth embodiment.
01, an error processing instruction signal Ecsel for instructing whether or not to interrupt decoding of the input video stream VOPstr upon detection of a transmission error or the like.
Is provided by an external instruction unit 701 generated by a user operation. Further, the decoding unit 102f is the switch control unit 304 of the decoding unit 102e of the fifth embodiment.
e, an error processing instruction signal Ecsel, a transmission error notification signal TRerr, and a stream error notification signal STerr
An error notification unit 304f that generates a switch control signal Coff based on the

Here, the error notifying section 304f outputs the transmission error notifying signal TRerr and the decoding error notifying signal STe.
When at least one of rr is to notify the occurrence of an error, and the error processing instruction signal Ecsel indicates that the decoding process for the input video stream VOPstr is interrupted when a transmission error or the like is detected. , And outputs a switch control signal Coff to the open / close switches 305 and 308 to make them non-conductive. Therefore, if no error has been detected,
Alternatively, if the error processing instruction signal Ecsel indicates that the decoding process on the input video stream VOPstr is not interrupted when a transmission error or the like is detected, the open / close switch 3 is turned on by the switch control signal Coff.
05 and 308 do not become non-conductive. However,
In this case, the open / close switch 308 is connected to the second comparator 307
Is controlled by a control signal Cpre from the controller.

The rest of the configuration of the decoding unit 102f according to the sixth embodiment is the same as the configuration of the decoding unit 102e according to the fifth embodiment. In FIG. 19,
The same reference numerals as those in FIG. 17 indicate the same components as those of the decoding unit 102e according to the fifth embodiment.

Next, the operation will be described. FIG. 20 is a diagram showing a flow of the operation of the decoding unit according to the sixth embodiment. In the decoding unit 102f according to the sixth embodiment,
The external instruction unit 701 determines in advance whether or not to suspend the decoding process when an error is detected, and the external instructing unit 701 determines whether or not to suspend the decoding process when an error is detected. Error processing instruction signal Ecse
l is output.

First, the decoding unit 10 of the fifth embodiment
As in 2e, a predetermined VO constituting the image sequence is set as a video stream VOPstr corresponding to one image sequence.
The VOP data of P is received by the decoding unit 102 (step S201), and subsequently, the second time extracting unit 301
In the received VOP data (the VO of the decoding target VOP)
The VOP display time Tin of (P data) is extracted (step S202).

Thereafter, in the decoding unit 102f according to the sixth embodiment, the switch control unit 304f determines whether or not the error processing instruction signal Ecsel performs a decoding process on the input video stream VOPstr when a transmission error or the like is detected. A determination is made as to whether or not the interruption is to be performed (step S701). As a result of this determination, when the error processing instruction signal Ecsel indicates that the decoding process for the video stream VOPstr is interrupted when a transmission error or the like is detected, the transmission error detection and the decoder 303 in the error detection unit are performed. Is detected at step S203. On the other hand, the above step S701
As a result of the determination in the above, when the error processing instruction signal Ecsel indicates that the decoding processing for the video stream VOPstr is not interrupted when a transmission error or the like is detected,
The error is not detected in step S203, and the second comparison unit 307 sets the VOP display time T of the decoding target VOP.
In is compared with the decrypted maximum time Tmax stored in the maximum time storage unit 306 (step S205). Thereafter, steps S204 and S2 in the fifth embodiment are performed.
The same processing as the processing from 06 to S208 is performed.

As described above, in the decoding unit 102f according to the sixth embodiment, when a transmission error or the like is detected, the error-time processing instruction signal Ecsel for instructing whether or not to interrupt the decoding processing for the video stream VOPstr is An external instruction unit 701 generated by a user's operation is provided. Based on the error processing instruction signal Ecsel, it is determined whether or not to interrupt the decoding process when an error is detected. It is possible to select whether or not to interrupt the decoding process upon detection.

(Embodiment 7) FIG. 21 is a block diagram for explaining a video decoding apparatus according to Embodiment 7 of the present invention, and shows a reordering unit constituting the video decoding apparatus. I have. Rearrangement unit 10 of the seventh embodiment
1g includes a first time extraction unit 201, a display time output unit 202, a pre-update time output unit 203, a first comparison unit 204, and a buffer memory 206, similarly to the reordering unit 101 of the first embodiment. I have. And this reordering unit 1
01g calculates the free space capacity (buffer remaining capacity) of the buffer memory 206 based on the information Iuse indicating the usage status of the buffer memory 206. When the buffer remaining capacity becomes equal to or less than a predetermined threshold, the data input limit is set. A buffer remaining amount calculation unit 801 that outputs a signal Irest;
And an open / close switch 802 provided at a stage preceding the control signal 06 to control the supply of the input coded stream Vin to the buffer memory 206 based on the data input restriction signal Irest. Also, the rearrangement unit 101g is the same as the selection switch 205 of the rearrangement unit 101 of the first embodiment.
In place of the control signal Csw from the first comparing unit 204.
Selection switch 205g that selects one of the VOP display time Tin and the pre-update display time Td ′ of the reception VOP based on the data input restriction signal Irest from the buffer remaining amount calculation unit 801 and outputs the same to the buffer memory 206. It has.

Here, the open / close switch 802 is connected to the data input restriction signal Irest from the buffer remaining capacity calculation unit 801.
Thus, a non-conducting state is established only when the remaining buffer amount is smaller than a predetermined threshold, and a conducting state is established when the remaining buffer amount is larger than a predetermined threshold value. That is, when the capacity of the free space of the buffer memory 206 is smaller than the predetermined threshold, the received VOP data
2 and is not output to the buffer memory 206 as the memory input data MEMin. On the other hand, the received V
The OP data is stored in the open / close switch 503 when the capacity of the free area of the buffer memory 206 is equal to or larger than a predetermined threshold.
Is output to the buffer memory 206 as memory input data MEMin.

When the remaining buffer amount Br is smaller than the threshold value Bth (Br <Bth), the selection time 205d is selected by the selection switch 205g. Conversely, when the remaining buffer amount Br is equal to or greater than the threshold value Bth (Br ≧ Bth). ), One of the VOP display time Tin and the pre-update display time Td ′ is selected by the control signal Csw1 from the first comparing unit 204. That is, when the VOP display time Tin of the reception VOP is a time after the display time before update Td ′ (Tin ≧ Td ′),
The display time before update Td 'is selected, and the display time before update T
d ′ is the decoding restart time Tstart
6 is output. On the other hand, the VOP display time T of the reception VOP
in is earlier than the display time before update Td ′ (Tin <T
In the case of d ′), the VOP display time Tin is selected, and the VOP display time Tin is output to the buffer memory 206 as the decoding restart time Tstart. In the seventh embodiment, a buffer remaining amount calculation unit 801 is provided instead of the oldest time extraction unit 501 and the fourth comparison unit 502 in the reordering unit 101c of the third embodiment. 2
1, the same reference numerals as those in FIG. 13 indicate the same components as those of the reordering unit 101c of the third embodiment.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the seventh embodiment is performed by
Except for the operation of g, the operation is the same as that of the video decoding apparatus according to the third embodiment.
The operation of g will be described.

FIG. 22 is a diagram showing a flow of the operation of the rearrangement unit 101g. First, similarly to the rearrangement unit 101c of the third embodiment, reception of VOP data input as an encoded stream Vin (step S101), and extraction of the VOP display time Tin of the received VOP (step S10)
2), output of the designated display time Td (step S103),
The display time before update Td ′ is output (step S104).

Thereafter, the buffer remaining amount calculation unit 801 of the rearrangement unit 101g calculates the buffer remaining amount Br based on the information Use indicating the use state of the buffer memory 206 (step S801), and sets the buffer remaining amount Br to a threshold value. This is compared with Bth (step S801).

As a result of this comparison, if the remaining buffer amount Br is smaller than the threshold value Bth, the open / close switch 80
2 becomes non-conductive, and the VOP data of the received VOP is discarded (step S402). At this time, the first
The control signal Csw from the comparison unit 204 indicates that the VOP display time Tin is earlier than the pre-update display time Td ', the selection switch 205 g The display time before update Td 'is selected by the data input restriction signal Irest.

On the other hand, if the result of the comparison in step S 801 is that the buffer remaining amount Br is equal to or larger than the threshold value Bth, the data input restriction signal Ire from the buffer remaining amount calculating unit 801 is output.
By st, the open / close switch 802 becomes conductive, and the VOP data of the received VOP is supplied to the buffer memory 206 as the memory input data MEMin, and is recorded in the buffer memory 206 (step S105). After that,
Similarly to the rearrangement unit 101c of the third embodiment, the VOP display time Tin and the pre-update display time T
d ′ (step S106), and VOP data output processing according to the comparison result (step S107,
S108) is performed.

As described above, the rearranging unit 1 of the seventh embodiment
01g, when the VOP data of the preceding VOP is received later than the VOP data of the succeeding VOP, as in the reordering unit 101 of the first embodiment, the VOP data from the preceding VOP to the display target VOP to be displayed at that time As the video stream VOPstr
, The decoding unit outputs V
When the OP data is input later than the VOP data of the succeeding VOP, the decoding process can be restarted from the VOP data of the preceding VOP.

Also, the rearranging unit 1 of the seventh embodiment
In 01g, the capacity of the free area (buffer remaining capacity) of the buffer memory 206 is calculated based on the information Iuse indicating the usage status of the buffer memory 206. A buffer remaining amount calculation unit 801 that outputs a signal Irest is provided, and only when the buffer remaining amount Br is equal to or greater than the threshold value Bth, the VO of the received VOP
Since the P data is supplied to the buffer memory 206, an overflow of the buffer memory 206 can be avoided.

(Eighth Embodiment) FIG. 23 is a block diagram for explaining a moving picture decoding apparatus according to an eighth embodiment of the present invention, and shows a permutation unit constituting the moving picture decoding apparatus. I have. Rearrangement unit 10 according to Embodiment 8
1h includes a first time extraction unit 201, a display time output unit 202, a pre-update time output unit 203, a first comparison unit 204, and a buffer memory 206, similarly to the rearrangement unit 101 of the first embodiment. I have. And this reordering unit 1
01h sets the VOP display time Tin of the reception VOP output from the first time extraction unit 201 to the display time output unit 106
The VOP display time Tin is compared with the designated display time Td from
ΔT from the specified display time Td (Tin <Td)
A fifth comparing unit 90 that outputs a control signal Csw5 according to whether (ΔT = Td−Tin) is within a predetermined time Tth.
1 is provided in the preceding stage of the buffer memory 206, and the fifth
And an open / close switch 902 for controlling the supply of the input coded stream to the buffer memory 206 based on the control signal Csw5 from the comparison unit 901. The rearrangement unit 101h is different from the rearrangement unit 101 of the first embodiment in that the selection switch 205 is replaced with the control signal Csw1 from the first comparison unit 204 and the control signal Csw5 from the fifth comparison unit 901. And the reception V
A selection switch 205h is provided for selecting one of the OP VOP display time Tin and the pre-update display time Td 'and outputting it to the buffer memory 206.

Here, the open / close switch 902 is connected to the fifth
Of the VOP display time Tin to the designated display time Td only when the time ΔT from the VOP display time Tin to the designated display time Td exceeds the predetermined time Tth, and when the elapsed time ΔT is equal to or less than the predetermined threshold Tth, At some point, it is in a conductive state. That is, the received VOP data is
When the time ΔT from the VOP display time Tin to the designated display time Td exceeds the predetermined time Tth, the data is discarded by the open / close switch 902 and is not output to the buffer memory 206 as the memory input data MEMin. On the other hand, when the time ΔT from the VOP display time Tin to the designated display time Td is within a predetermined time Tth, the received VOP data is output to the buffer memory 206 as the memory input data MEMin via the open / close switch 902. Is done.

In the selection switch 205h, V
Time ΔT from OP display time Tin to designated display time Td
Exceeds the predetermined time Tth, the first comparison unit 2
The display time before update Td ′ is selected regardless of the control signal Csw1 from 04, and the display time before update Td ′ is output to the buffer memory 206 as the decoding restart time Tstart.
On the other hand, when the time ΔT from the VOP display time Tin to the designated display time Td is within the fixed time Tth, the selection switch 205h sets the VOP display time according to the control signal Csw1 from the first comparison unit 204. The early time of the Tin and the pre-update display time Td ′ is the decoding restart time Tstar.
selected as t.

Note that the eighth embodiment is different from the third embodiment.
Oldest time extracting unit 50 in the sorting unit 101c
Instead of the first and fourth comparison units 502, a fifth comparison unit 90
In FIG. 23, the same reference numerals as those in FIG. 13 indicate the same components as those of the reordering unit 101c of the third embodiment.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the eighth embodiment is the same as that of the reordering unit 101.
Except for the operation of h, the operation is the same as that of the video decoding apparatus of the third embodiment.
The operation of g will be described. FIG. 24 is a diagram showing a flow of the operation of the rearrangement unit 101h.

First, the reordering unit 101 according to the third embodiment
VO input as an encoded stream Vin as in the case of c.
P data reception (step S101), V of reception VOP
The OP display time Tin is extracted (step S102), the designated display time Td is output (step S103), and the pre-update display time Td 'is output (step S104). Thereafter, in the fifth comparing unit 901 of the rearrangement unit 101h, it is determined whether or not the time ΔT from the VOP display time Tin to the designated display time Td exceeds a certain time Tth (step S901).

As a result of this determination, the time ΔT (= Td−Ti
If n) exceeds the predetermined time Tth, the open / close switch 9 is turned on by the control signal Csw5 from the fifth control unit 901.
02 is turned off, and the VOP data of the received VOP is discarded (step S402). At this time, even if the control signal Csw1 from the first comparing unit 204 indicates that the VOP display time Tin is earlier than the pre-update display time Td ′, the selection switch 205h
Then, the display time before update Td ′ is selected by the control signal Csw5 from the fifth control unit 901.

On the other hand, if the result of the determination in step S901 is that the time ΔT (= Td−Tin) is within the fixed time Tth, the control signal Csw5 from the fifth control unit 901 is obtained.
As a result, the open / close switch 902 becomes conductive, and the VOP data of the received VOP is supplied to the buffer memory 206 as the memory input data MEMin.
06 (step S105).

Thereafter, the rearranging unit 1 of the third embodiment
01c, the first comparison unit 204 compares the VOP display time Tin with the pre-update display time Td ′ (step S
106), and output processing of VOP data according to the comparison result (steps S107 and S108).

As described above, the rearranging unit 1 of the eighth embodiment
01h, when the VOP data of the preceding VOP is received later than the VOP data of the succeeding VOP, the VOP data from the preceding VOP to the display target VOP to be displayed at that time is received in the same manner as the rearrangement unit 101 of the first embodiment. As the video stream VOPstr
, The decoding unit outputs V
When the OP data is input later than the VOP data of the succeeding VOP, the decoding process can be restarted from the VOP data of the preceding VOP.

Further, the rearranging unit 1 of the eighth embodiment
01h, the VOP data of the received VOP is transmitted only when the time difference between the VOP display time of the delayed VOP received late and the designated display time Td on the decoding side at the time of receiving the delayed VOP is within a fixed time. Since the VOP is supplied to the buffer memory 206, the VOP from the delayed VOP to the display target VOP is
When a part of the data is deleted from the buffer memory 206 and the VOP data from the delayed VOP to the display target VOP cannot be decoded normally, the encoded data of the delayed VOP is prevented from being output to the buffer memory. be able to.

(Embodiment 9) FIG. 25 is a block diagram for explaining a video decoding apparatus according to Embodiment 9 of the present invention, and shows a reordering unit constituting the video decoding apparatus. I have. Rearrangement unit 10 according to Embodiment 9
1i, each VO from the input coded stream Vin
A first time extraction unit 201 for extracting a VOP display time Tin set for P and a timer (not shown) are built in, and a moving image decoding is performed based on reference time information in the encoded stream Vin. A display time output unit 202 for updating and outputting a designated display time Td for designating a VOP display timing on the device 100 side, and the encoded stream Vin
And a buffer memory 206 for storing

[0220] The rearranging unit 101i also updates the VOP display time Tin of the reception VOP (the VOP at which the VOP data was received) and the setting display time Td0 set at the time of reception of the reception VOP before the update ( A first subtractor 1120 that calculates a difference value (Td0′−Tin) from the setting display time before update (Td0 ′), and a delay notification signal (startSw) indicating that VOP data of a predetermined VOP has been received with a delay.
Signal) Ndel, and outputs the difference value (Td0'-Tin) as the correction time Tdd.
Every time the designated display time Td is updated, the subtraction counter 111 that gradually reduces the value of the output correction time Tdd
0. Here, the subtraction counter 1110
Outputs 0 as the value of the correction time after the value of the correction time Tdd to be output becomes 0, and then again outputs the predetermined VO
When receiving a delay notification signal (startSw signal) Ndel indicating that the P VOP data has been received late, the correction time Tdd to be output is gradually reduced from the difference value (Td0'-Tin). .

Further, the rearrangement unit 101i calculates the correction time Td output from the subtraction counter 1101 from the designated display time Td output from the display time output unit 202.
A second subtractor 1130 that outputs a time Td0 obtained by subtracting d as the set display time, and outputs a value before update (set display time before update) Td0 ′ based on the set display time Td0. Pre-update time output unit 203i
And a delay determination unit 204 that outputs the delay notification signal (startSw signal) Ndel by comparing the VOP display time Tin from the first time extraction unit 201 with the pre-update setting display time Td0 ′. A selection switch 205i for selecting one of the VOP display time Tin and the pre-update setting display time Td0 'based on the notification signal (startSw signal) Ndel, and outputting the selected time as the decoding restart time Tstart. I have.

Here, delay determining section 204i determines whether or not the received VOP from which the VOP data has been received is a delayed VOP from which the VOP data has been received with a delay from the reception timing at which the decoding processing can be performed. , The receiving VOP is delayed V
When there is an OP, the delay notification signal (startSw signal) Ndel
Is output. Also, the selection switch 205i
Means that the delay notification signal (startSw signal) Ndel is
When the OP indicates the delay VOP, the VOP display time Tin is selected, and the delay notification signal (startSw signal) N
When del indicates that the received VOP is not a delayed VOP, the setting display time before update Td0 'is selected. Further, the buffer memory 206 stores the VOP VOP data whose VOP display time is within the range from the decoding restart time Tstart from the selection switch 205i to the set display time Td0 from the second subtractor 1103. It is output as VOPstr.

Next, the operation will be described. First, when the coded stream Vin corresponding to one image sequence (object) is received by the reordering unit 101i, the VOP data received as the coded stream Vin is stored in the buffer memory 2
06. In the first time extracting unit 201, the VOP display time corresponding to the VOP data received as the encoded stream Vin (the VOP display time of the received VOP)
Tin is extracted. In the display time output unit 202,
Based on the reference time information included in the encoded stream Vin, the designated display time Td indicating the display timing of the VOP on the video decoding device side is updated and output.

At this time, in the first subtractor 1120, the VOP of the received VOP (the VOP in which the VOP data was received)
A difference value (Td0'-Tin) between the display time Tin and a value (Td0'-Tin) before the update (setting display time before update) Td0 'of the setting display time Td0 set at the time of receiving the reception VOP is calculated. (Td0′−Tin) is the subtraction counter 111
Output to 0. In the subtraction counter 1110, a predetermined V
When a delay notification signal (startSw signal) Ndel indicating that the VOP data of the OP has been received late is input from the delay determination unit 204i, the difference value (Td0'-Tin) is set as the correction time Tdd and the second subtractor 1130. After the difference value (Td0′−Tin) is output as the correction time Tdd, each time the display time output unit 202 updates the designated display time Td, the output correction time Tdd gradually decreases. . After the value of the correction time Tdd becomes 0,
Until the delay notification signal (startSw signal) Ndel is input,
The value of the correction time Tdd is maintained at zero.

On the other hand, when the delay notification signal (startSw signal) Ndel indicating that the VOP data of the predetermined VOP has been received late is not input,
Difference value (Td0'-Tin) from first subtractor 1120
Regardless of this, 0 is output to the second subtractor 1130 as the value of the correction time Tdd.

In the second subtractor 1130, a time Td0 obtained by subtracting the correction time Tdd output from the subtraction counter 1101 from the designated display time Td output from the display time output unit 202 is set as the set display time. Is output. The pre-update time output unit 203i outputs the pre-update value (pre-update setting display time) Td0 'based on the setting display time Td0. The delay determination unit 204 outputs the delay notification signal (startSw signal) Ndel by comparing the VOP display time Tin from the first time extraction unit 201 with the setting display time before update Td0 ′. That is, the delay determination unit 204i determines whether or not the reception VOP that has received the VOP data is a delay VOP that has received the VOP data later than the reception timing at which the decoding processing can be performed. When there is a delayed VOP,
The delay notification signal (startSw signal) Ndel is output.

The selection switch 205i selects one of the VOP display time Tin and the pre-update setting display time Td0 ′ based on the delay notification signal (startSw signal) Ndel, and the selected time is used as the decoding restart time Tstart. The data is output to the buffer memory 206. That is, select switch 2
05i, the delay notification signal (startSw signal) Ndel
Indicates that the received VOP is a delayed VOP,
The OP display time Tin is selected, and the delay notification signal (star
When the tSw signal) Ndel indicates that the received VOP is not a delayed VOP, the pre-update setting display time Td0 'is selected.

In the buffer memory 206,
The VOP of the VOP whose VOP display time is within the range from the decoding restart time Tstart from the selection switch 205i to the set display time Td0 from the second subtractor 1103
Data is output as a video stream VOPstr.

Next, the effect of the ninth embodiment will be described. In the rearrangement unit 101i of the ninth embodiment,
When the VOP data of a predetermined VOP is received with a delay,
A predetermined number of succeeding VOPs following the predetermined VOP are displayed such that the actual display time that is later than the original display time of each VOP gradually approaches the original display time. Even if there is a VOP, the display time of the succeeding VOP following the delayed VOP is gently corrected, and a moving image with a smooth motion can be displayed.

That is, in a VOP received with a delay, the actual display time (set display time Td0) is smaller than the original display time (designated display time Td) by the time Tdd corresponding to the delay time. Therefore, the buffer memory 2
From 06, the VOP display time is changed to the original display time Td.
The VOP data of the VOP (delayed VOP) that is earlier than the time is output, and the delayed VOP is displayed.

Hereinafter, a case where the VOP data of VOP (i) is received with a delay will be specifically described. FIG.
(a) shows the succeeding VOP (i + 1) following the delay VOP (i)
To (i + 3) actual display time (setting display time Td0)
Indicates that the display time gradually approaches the original display time (designated display time Td). The delay VOP (i) is a time t 'delayed from the original display time t (i) by Tdd (= Tdd0).
Subsequent VOP displayed at (i) and following delayed VOP (i)
(I + 1) is equal to T from its original display time t (i + 1).
Time t '(i + 1) delayed by dd (= Tdd1 <Tdd0)
Will be displayed. Further, the succeeding VOP (i + 2) is Tdd (= Tdd2 <Tdd) from its original display time t (i + 2).
1) delayed at time t ′ (i + 2), and the subsequent VOP (i + 3) is displayed at its original display time t (i + 2).
It is displayed at the same time t ′ (i + 3) as in 3).

In this case, VOP (i), VOP (i +
1), VOP (i + 2), VOP (i + 3)
The difference value Tdd (= Td-Td) between the actual display time (setting display time Td0) and the original display time (designated display time Td).
0) is reduced by the subtraction counter 1110, and in the VOP (i + 3), the actual display time Td0 matches the original display time subtraction Td, and the display time of the subsequent VOP is delayed due to the occurrence of the delay VOP. Completely disappears. On the other hand, if the display time of the subsequent VOP following the delay VOP is not adjusted, the subsequent VOP will be displayed at the original display time, and if a delay VOP occurs, the movement is temporarily unnatural. The video will be displayed.

FIG. 26 (b) shows how subsequent VOPs (i + 1) to (i + 3) following the delayed VOP (i) are displayed at the original display time (designated display time Td). Delay VOP (i) is Tdd from its original display time t (i).
(= Tdd0), it is displayed at time t ′ (i), but following VOP (i + 1),
(I + 2) and (i + 3) are displayed at their original display times t (i + 1), t (i + 2), and t (i + 3), respectively.

As described above, in the ninth embodiment, similar to the rearrangement unit 101 of the first embodiment, the VOP of the preceding VOP is
When data is received later than the VOP data of the succeeding VOP, the display target V to be displayed at that time from the preceding VOP
The VOP data up to the OP is converted into a video stream VOPst
r is output to the decoding unit 102, so that the decoding unit replaces the VOP data of the preceding VOP with the succeeding VOP.
Is input later than the VOP data of
The decoding process can be restarted from the VOP data of the OP.

When the VOP data of a predetermined VOP is received with a delay, the difference between the actual display time (set display time Td0) of the succeeding VOP following this and the original display time (designated display time Td) is obtained. Value Tdd (= Td-Td0)
Is provided with a subtraction counter 1101 that makes the display time smaller as the original display time is later. Therefore, even if there is a delay VOP received with a delay in VOP data, the display time of the subsequent VOP following the delay VOP is gently corrected. Will be
You can display smooth moving videos.

(Embodiment 10) FIG. 27 is a block diagram for explaining a moving picture decoding apparatus according to Embodiment 10 of the present invention, and shows a reordering unit constituting the moving picture decoding apparatus. I have. The reordering unit 101j in the video decoding device according to the tenth embodiment is the
The first time extracting unit 20
1, display time output section 202, pre-update time output section 203,
A first comparison unit 204, a selection switch 205, and a buffer memory 206 are provided.

Then, the rearrangement unit 101j is connected to the input coded stream Vin and the selection switch 2
An unreceived data determination unit 1201 that outputs a control signal Cde based on the decoding restart time Tstart output from the control unit 05,
It is provided between the buffer memory 206 and the selection switch 205 and supplies the decoding restart time Tstart output from the selection switch 205 to the buffer memory 206 based on the control signal Cde from the unreceived data determination unit 1201. And an open / close switch 1202 for control.

Here, the unreceived data determination unit 1201
The VOP display time is a time between the VOP display time Tin of the reception VOP and the decoding restart time Tstart, and the VOP data is not received (unreceived VO).
P) is determined, and a signal indicating the determination result is output as the control signal Cre. The open / close switch 1202 is turned off when the control signal Cre indicates that there is an unreceived VOP, and turned on when the control signal Cre indicates that there is no unreceived VOP. That is, when an unreceived VOP occurs, the decoding restart time Tstart is not supplied to the buffer memory 206, and the video stream VO is output from the buffer memory 206.
When Pstr is not output and the VOP data of the unreceived VOP is received thereafter, the buffer memory 206
The VOP data from the delayed VOP to the display target VOP received late is output as a video stream VOPstr.

Next, the operation will be described. The operation of the moving picture decoding apparatus according to the tenth embodiment is
1j is the same as the operation of the video decoding apparatus of the first embodiment except for the operation of 1j.
The operation of 1j will be described.

FIG. 28 is a diagram showing a flow of the operation of the rearrangement unit 101j. First, similarly to the reordering unit 101 of the first embodiment, reception of VOP data input as an encoded stream Vin (step S101) and extraction of a VOP display time Tin of the received VOP (step S10)
2), output of the designated display time Td (step S103),
The display time before update Td ′ is output (step S104), and the encoded stream Vin is recorded in the buffer memory 206 (step S105).

Thereafter, in the unreceived data determination section 1201 of the rearrangement unit 101j, the VOP display time is set to the reception VOP based on the input coded stream Vin and the decoding restart time Tstart output from the selection switch 205. From the VOP display time Tin to the decoding restart time Tstar
It is determined whether or not there is a VOP that is a time during t and for which no VOP data has been received (unreceived VOP) (step S1201).

At this step S1201, unreceived VOP
If it is determined that there is an open / close switch 1202,
The control signal Cre turns off and the decoding restart time T
start is not supplied to the buffer memory 206. For this reason, the video stream V
OPstr is not output. On the other hand, if it is determined in step S1201 that there is no unreceived VOP, the open / close switch 1202 is turned on by the control signal Cre, and the decoding restart time Tstart is supplied to the buffer memory 206.

Thereafter, the rearranging unit 1 of the first embodiment
01, the VOP display time T in the first comparison unit 204
in and the display time before update Td ′ (step S1)
06), and VOP data output processing according to the comparison result (steps S107 and S108). That is,
If the data VOP display time Tin is earlier than the display time before update Td ′ (Tin <Td ′), the VOP display time is changed to the reception VOP VOP display time Tin and the designated display time T.
The VOP data of the VOP which is the time between d is output from the buffer memory 206 (step S108). On the other hand, as a result of the comparison in step S105, the VOP display time Tin is a time after the pre-update display time Td '(T
in ≧ Td ′), the VOP display time is the time between the display time before update Td ′ and the specified display time Td.
OP data is output from the buffer memory 206 (step S106).

As described above, in the tenth embodiment, the VO of the preceding VOP is similar to the rearrangement unit 101 of the first embodiment.
When the P data is received later than the VOP data of the succeeding VOP, the VOP data from the preceding VOP to the display target VOP to be displayed at that time is transferred to the video stream VOP.
Since it is output to the decoding unit 102 as str, the decoding unit replaces the VOP data of the preceding VOP with the succeeding VO.
When the data is input later than the P VOP data, the decoding process can be restarted from the VOP data of the preceding VOP.

In the tenth embodiment, the VOP display time is a VOP which is a time between the VOP display time Tin of the received VOP and the decoding restart time Tstart, and
An unreceived data determination unit 1201 is provided for determining whether there is a VOP for which P data has not been received (unreceived VOP). When it is determined that there is an unreceived VOP, the received VOP is determined.
The output of the VOP data from the VOP to the display target VOP to the decoding unit is stopped, and when it is determined that there is no unreceived VOP, the VOP data from the received VOP to the display target VOP is output to the decoding unit. If there is a received VOP, the output of the video stream VOPstr from the buffer memory to the decoding unit is stopped until this is received. As a result, even when the decoding unit is composed of only the decoder that performs the decoding process on the input video stream VOPstr, the inter-frame prediction decoding process is performed without the correct reference image data. This can prevent image quality degradation due to the presence of an unreceived VOP, and prevent image quality degradation once occurring in a predetermined VOP from being carried over to subsequent VOPs.

As the reordering unit 101j of the tenth embodiment, the unreceived data determination unit 1201 and the open / close switch 1 in the reordering unit 101 of the first embodiment.
202 is shown, the second and third embodiments are described.
4, 7, 8, 9 rearranging units 101b, 101c, 1
01d, 101g, 101h, and 101h may include the unreceived data determination unit 1201 and the open / close switch 1202.

Further, the image decoding apparatus shown in each of the above embodiments may be realized by software instead of hardware. For example, a decoding program for causing a computer to perform the operation of the image decoding apparatus is recorded on a program recording medium, and the decoding program is loaded from the program recording medium into a computer system, thereby decoding the image shown in each embodiment. Can be realized by software.

The following is a brief description. FIG. 29 is an explanatory diagram when a floppy (registered trademark) disk is used as the program recording medium. FIG. 29A is a diagram for explaining the physical format of a floppy disk, and FIG. 29B shows a floppy disk and a floppy disk case for accommodating the same. The floppy disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery on the surface of the disk, and each track is divided into 16 sectors Se in an angular direction. Have been. In the floppy disk storing the decryption program, data as the decryption program is recorded in an area allocated on the floppy disk FD.

FIG. 29C shows an example of a specific configuration for writing and reading a decryption program on the floppy disk FD. The decryption program is written from the computer system Cs to the floppy disk FD via the floppy disk drive FDD. The decryption program recorded on the floppy disk FD is read out to the computer system Cs via the floppy disk drive FD. In the computer system Cs, the moving picture decoding apparatus according to each of the above-described embodiments is constructed by the decoding program loaded from the floppy disk FD.

In the above description, a floppy disk is used as the program recording medium. However, the program recording medium is not limited to a floppy disk. For example, an optical disk, hard disk, magnetic disk, magneto-optical disk, ROM, nonvolatile memory It may be a card or the like.

[0251]

As described above, the present invention (Claims 1, 7,
According to 11), encoded data of each image constituting a moving image is received as an encoded stream from the transmission side,
A reordering step of performing a reordering process of transmitting the received encoded data of each image in order from the earliest individual time set for each image to a transmission time corresponding to each image, and When the encoded data of the image to be transmitted at the time is received later than the second transmission time, which is later than the first transmission time, the individual time is changed from the first transmission time to the second transmission time.
Since the encoded data of the image that is within the range up to the transmission time is sent out, when the encoded data of the image is received with a delay, of the encoded data of the image received earlier,
The data necessary for decoding the image data to be displayed at the time of reception is sent out. Therefore, in the decoding step of performing the decoding process of the transmitted encoded data, the decoding process of the encoded data can be started from the encoded data of the image received late, Even if the encoded data of the image is received late,
It is possible to perform decoding processing on encoded data without causing deterioration in image quality to occur.

According to the present invention (claim 2), claim 1
In the moving picture decoding method described, the encoded data of the picture constituting the moving picture is received when the display order constituting the moving picture is received earlier than the time at which the last picture to be displayed is last. When the encoded data of the image is transmitted and the encoded data of the image constituting the moving image is received after the time at which the final image is to be displayed, the encoded data of the received image is discarded. The coded data of the image that is no longer required to be converted is discarded. Therefore, the buffer memory for storing the encoded data of the image can be effectively used, and the number of image sequences (objects) that can be simultaneously decoded by the decoding processing unit can be prevented from being unnecessarily increased.

According to the present invention (claim 3), claim 1
In the moving picture decoding method described above, when the encoded data of the image to be transmitted at the first transmission time is received later than the second transmission time later than the first transmission time, the individual time is set to the first transmission time. If it is not possible to transmit the encoded data of all the images within the range from the transmission time to the second transmission time, the encoded data of the received image is discarded, and there is no correct reference image data. In this state, it is possible to avoid performing the inter-picture prediction decoding process that refers to the decoded image data, thereby preventing the propagation of image quality deterioration in the decoding process.

According to the present invention (claims 4, 9 and 12), coded data of each picture constituting a moving picture is received as a coded stream from the transmission side, and the coded data of each received picture is And a rearrangement step of performing a rearrangement process of sequentially sending out the individual time set in each image in an order from the earliest one. The encoded data of an image for which normal decoding cannot be performed by the rearrangement process is included. When transmitted, from the time of transmission of the encoded data, thereafter, by the above rearrangement processing,
Until the encoded data of the image that can be normally decoded is transmitted, the decoding of the encoded data of the image is interrupted, so that it is possible to avoid a significant deterioration in the image quality of the image, and furthermore, to prevent the image from being degraded. It is possible to prevent the propagation of image quality deterioration in which the state of image quality deterioration continues.

According to the present invention (Claim 5), Claim 4
In the moving picture decoding method described above, whether or not to interrupt the decoding process when encoded data of an image for which the normal decoding process is impossible is transmitted is determined based on a user operation signal. Since the determination is made, it is possible to select whether or not to interrupt the decoding process when an error in the encoded data is detected, according to the user's preference and application.

According to the present invention (Claim 6), Claim 4
In the moving picture decoding method described above, only when the encoded data of the image includes shape information, when the encoded data of the image that cannot be normally decoded is transmitted, the encoded data is Since the decoding process for the coded data is interrupted from the time of transmission of the coded data until the coded data of the image that can be normally decoded is transmitted, adverse effects on the image quality such as transmission errors are prevented. small,
For an image sequence having no shape signal, a decoding process can be performed so as not to impair the real-time property, and furthermore, for an image sequence having a shape signal, which has a large adverse effect on image quality such as a transmission error. Can reduce image quality degradation due to transmission errors and the like.

According to the present invention (claim 8), claim 7
In the moving picture decoding apparatus according to the above aspect, in the display image output unit, the data rearrangement unit encodes the encoded data of the image whose individual time is a time within the range from the first transmission time to the second transmission time. Is transmitted, out of the encoded data of the transmitted image, only the decoded image data of the image whose individual time is closest to the second transmission time is output as the display image data. In addition, it is possible to always display the newest image among the decoded images.

According to the present invention (Claim 10), in the moving picture decoding apparatus according to the ninth aspect, in the display image output section, the decoding section for the encoded data of the image is interrupted by the decoding section. During this time, the decoded image data that has been subjected to the decoding process last in the decoding unit is output as display image data. Therefore, even during the interruption of the decoding process for the encoded data, the correct decoding process is performed. The displayed image can be displayed.

According to the present invention (claims 13, 17 and 20), when receiving coded data of each image constituting a moving image and performing decoding processing of the coded data, a required transmission time is set. If the encoded data of the image to be sent to the decoding unit cannot be received by the required transmission time, the decoding of the encoded data of the image is performed until the encoded data of the image to be sent at the required transmission time is received. After the transmission to the decoding unit is stopped and the encoded data of the image to be transmitted at the required transmission time is received, the transmission of the encoded data of the image to the decoding unit is restarted. If there is coded data of an unreceived image that has not been received, the output of coded data to the decoding unit is stopped until this is received. As a result, even when the decoding unit includes only a decoder that performs a decoding process on the input coded data, the inter-picture prediction decoding process is performed without correct reference image data. This can prevent image quality degradation due to the presence of encoded data of an unreceived image, and prevent image quality degradation once occurring in a predetermined image from being carried over to subsequent images. be able to.

According to the present invention (claims 14, 18, and 21), when receiving the encoded data of each image constituting the moving image and performing the decoding process on the encoded data, the required transmission time If the encoded data of the image to be transmitted cannot be received by the required transmission time, the image data is transmitted until the predetermined condition is satisfied or the encoded data of the image to be transmitted at the required transmission time is received. The transmission of the encoded data is stopped, and after the predetermined condition is satisfied, or after the encoded data of the image to be transmitted at the required transmission time is received, the transmission of the encoded data of the image is restarted. If there is encoded data of an unreceived image that has not been received by the corresponding transmission time, the encoded data to the decoding unit is received until this is received or a predetermined condition is satisfied. So that the output of is stopped. As a result, it is possible to prevent the image quality from deteriorating due to the presence of encoded data of an unreceived image, and to prevent the operation of the buffer memory from being hindered by unnecessary encoded data.

According to the present invention (claim 15), in the moving picture decoding method according to claim 14, the predetermined condition is set such that a coded stream of each picture received at the time of the rearrangement processing is stored. Since the condition is such that the remaining storage capacity of the buffer memory is equal to or less than the predetermined value, overflow of the buffer memory can be avoided.

According to the present invention (claim 16), in the moving picture decoding method according to claim 14, the predetermined condition is set such that transmission of coded image data is stopped during the rearrangement process. Time is equal to or greater than the predetermined value,
A part of the encoded data from the delayed image whose arrival of the encoded data is delayed to the image to be displayed at that time has been deleted from the buffer memory, and the encoded data from the delayed image to the image to be displayed is deleted. If decryption is not successful,
The encoded data of the delayed image can be prevented from being output to the buffer memory.

According to the present invention (claim 19), in the moving picture decoding apparatus according to claim 18, after the transmission of the encoded data of the picture to the decoding unit is stopped by the data rearranging unit. When the transmission of the encoded data of the image is restarted, the image display within a certain period is performed so that the delay of the display timing of the image caused by the stop of the transmission becomes smaller each time the image is displayed. If the coded data of an image is received with a delay, a predetermined number of succeeding images following the predetermined image are gradually displayed with the actual display time delayed from the original display time of each image. It is displayed as approaching the time. For this reason, even if there is a delayed image whose encoded data is received with a delay, the display time of the image following the delayed image is gently corrected, and a moving image with smooth motion can be displayed.

[Brief description of the drawings]

FIG. 1 is a video decoding device 1 according to a first embodiment of the present invention.
It is a block diagram for explaining 00.

FIG. 2 is a diagram illustrating a flow of processing of an encoded stream in the video decoding device 100 according to the first embodiment.

FIG. 3 is a block diagram for explaining a reordering unit 101 included in the video decoding device 100 according to the first embodiment.

FIG. 4 is a flowchart illustrating an operation of the rearrangement unit 101 according to the first embodiment.

FIG. 5 is a block diagram for explaining a decoding unit 102 included in the video decoding device 100 according to the first embodiment.

FIG. 6 is a flowchart showing an operation of the decoding unit 102 according to the first embodiment.

FIG. 7 is a diagram for explaining another example of the processing of the encoded stream in the video decoding device 100 according to the first embodiment.

FIG. 8 shows another example of the permutation unit (the permutation unit 101a) included in the moving picture decoding apparatus 100 according to the first embodiment.
It is a block diagram for explaining.

FIG. 9 is a flowchart illustrating an operation of the rearrangement unit 101a according to the first embodiment.

FIG. 10 is a block diagram for explaining a video decoding device according to a second embodiment of the present invention, and shows a reordering unit constituting the video decoding device.

FIG. 11 is a flowchart illustrating an operation of a rearrangement unit 101b according to the second embodiment.

FIG. 12 is a diagram for explaining the operation of the video decoding device according to the second embodiment, and shows six image sequences (objects A to A);
F) is displayed (FIG. (A)), and the coded stream of each image sequence is converted to each reordering unit (decoding unit).
Indicates the time zone to be processed.

FIG. 13 is a block diagram for explaining a video decoding device according to a third embodiment of the present invention, and shows a reordering unit 101c included in the video decoding device.

FIG. 14 is a flowchart illustrating an operation of a rearrangement unit 101c according to the third embodiment.

FIG. 15 is a block diagram for explaining a video decoding device according to a fourth embodiment of the present invention, and shows a reordering unit 101d included in the video decoding device.

FIG. 16 is a flowchart illustrating an operation of a rearrangement unit 101d according to the fourth embodiment.

FIG. 17 is a block diagram for explaining a video decoding device according to a fifth embodiment of the present invention, and shows a decoding unit 102e included in the video decoding device.

FIG. 18 is a flowchart illustrating an operation of the decoding unit 102e according to the fifth embodiment.

FIG. 19 is a block diagram for explaining a video decoding device according to a sixth embodiment of the present invention, and shows a decoding unit 102f constituting the video decoding device.

FIG. 20 is a flowchart showing an operation of the decoding unit according to the sixth embodiment.

FIG. 21 is a block diagram for explaining a video decoding device according to a seventh embodiment of the present invention, and shows a reordering unit 101g constituting the video decoding device.

FIG. 22 is a flowchart illustrating an operation of a rearrangement unit 101g according to the seventh embodiment.

FIG. 23 is a block diagram for explaining a video decoding device according to Embodiment 8 of the present invention, and shows a reordering unit 101h included in the video decoding device.

FIG. 24 is a flowchart illustrating an operation of a rearrangement unit 101h according to the eighth embodiment.

FIG. 25 is a block diagram for explaining a video decoding device according to a ninth embodiment of the present invention, and shows a reordering unit 101i included in the video decoding device.

FIG. 26 is a diagram for explaining the effect of the ninth embodiment, in which the actual display time Td0 of the succeeding VOP (i + 1) to (i + 3) following the delay VOP (i) gradually becomes the original display time. A state approaching the display time Td (FIG. 13A) and a subsequent VO following the delay VOP (i)
P (i + 1) to (i + 3) are the original display time (designated display time Td)
(Fig. (B)).

FIG. 27 is a block diagram for explaining a moving picture decoding apparatus according to Embodiment 10 of the present invention, and shows a reordering unit 101j constituting the moving picture decoding apparatus.

FIG. 28 is a flowchart showing an operation of a rearrangement unit 101j according to the tenth embodiment.

FIG. 29 shows a recording medium (FIGS. 29A and 29B) storing a decoding program for realizing the moving picture decoding apparatus of each of the embodiments by a computer system, and the computer system shown in FIG. FIG.

FIG. 30 is a block diagram illustrating a conventional video decoding device.

FIG. 31 is a diagram for explaining the flow of processing of an encoded stream in a conventional video decoding device, in which the order of VOP data in a received encoded stream is correct (FIG. 31A); When the order of the VOP data in the received encoded stream is not correct (FIG.
(b)).

[Fig. 32] Fig. 32 is a diagram for describing the flow of processing of an encoded stream in a conventional video decoding device.

[Explanation of symbols]

100 video decoding device 101, 101a, 101b, 101c, 101d, 1
01g, 101h, 101i, 101j Rearrangement unit 102, 102e, 102f Decoding unit 103 Video memory 104 Video compositor unit 105 Display unit 201 First time extraction unit 201a Intra VOP time extraction unit 202 Display time output unit 203, 203i Pre-update time output unit 204 first comparison unit 204i delay determination unit 205, 205c, 205g, 205h, 205i selection switch 206 buffer memory 301 second time extraction unit 302 error detection unit 303 decoder 304, 304e error notification unit 305, 308,403,503,802,902,1
202 Open / close switch 306 Maximum time holding unit 307 Second comparing unit 401 Valid time extracting unit 402 Third comparing unit 501 Earliest time extracting unit 502 Fourth comparing unit 601 Shape existence determining unit 701 External instruction unit 801 Buffer remaining capacity Calculation unit 901 Fifth comparison unit 1001 Reordering unit 1002 Decoding unit 1003 Video memory 1004 Video compositor unit 1005 Display unit 1101 Memory 1102 Display time output unit 1103 Decoder 1110 Subtraction counter 1120 First subtractor 1130 Second subtraction Detector 1201 Not received, judgment part Cs Computer system Csw1, Cde, Cpre, Csw3, STRsw, Csw5 Control signal Ecsel External operation signal FD Floppy disk FDD Floppy disk drive F Floppy disk case Itime Time information Irest Buffer remaining information Iuse Memory usage information MEMin Memory input data Nerr Error notification signal Ndel Delay determination signal Se Sector Shape Shape determination signal Tr Track Td Specified display time Td ', Td0' Display time before update Td0 Setting display time Tdd counter Output (difference time) Tin VOP display time Tstart Decoding restart time TRerr Transmission error notification signal TSerr Stream error notification signal Tmax Maximum time Tmin Oldest time Tivop Intra VOP display time Tval Effective time Vin Encoded stream VOPstr Video stream VOPref Reference image data VOPout Decoded image data VOPdsp Display image data VOPsel Display screen selection signal Vout Composite image data

Claims (21)

[Claims] 1. A moving picture decoding method for decoding a coded stream obtained by coding image data of a moving picture composed of images for which individual times are set, wherein the coded stream is provided as a coded stream from a transmission side. Performs a rearrangement process of receiving the encoded data of each image and sending out the received encoded data of each image to the transmission time corresponding to each image in order from the earliest individual time set for each image. A rearrangement step; and a decoding step of performing a decoding process on the encoded data of each image transmitted by the rearrangement process to generate decoded image data corresponding to each image. The step is such that, when the encoded data of the image to be sent at the first sending time is received later than the second sending time that is later than the first sending time, the individual time is changed from the first sending time to the second sending time. A moving image decoding method for transmitting encoded data of an image having a time within a range up to a transmission time. 2. The moving picture decoding method according to claim 1, wherein the rearrangement step includes the steps of: encoding the encoded data of the picture constituting the moving picture, and converting the encoded data of the picture constituting the moving picture into the last picture in the last display order. Is received before the time it should be displayed,
When the encoded data of the received image is transmitted and the encoded data of the image constituting the moving image is received after the time at which the final image should be displayed, the encoded data of the received image is discarded. A moving picture decoding method, characterized in that: 3. The moving picture decoding method according to claim 1, wherein the rearranging step includes the step of: converting encoded data of an image to be sent at the first sending time to a second sending time later than the first sending time.
If the coded data of all the images whose individual time is within the range from the first transmission time to the second transmission time can be transmitted when the data is received later than the transmission time of If the encoded data of all the images whose individual time is within the range from the first transmission time to the second transmission time cannot be transmitted, the code of the received image is transmitted. A moving picture decoding method for discarding encoded data. 4. A moving picture decoding method for decoding a coded stream obtained by coding image data of a moving picture composed of images each having an individual time set, wherein the coded stream is provided as a coded stream from a transmission side. A reordering step of receiving encoded data of each image, and performing a reordering process of sequentially sending the received encoded data of each image in ascending order of the individual time set for each image; Performing a decoding process on the encoded data of each image sent by the above, to generate decoded image data corresponding to each image, the decoding step, by the reordering process, When coded data of an image for which normal decoding cannot be performed is transmitted, from the point in time at which the coded data is transmitted, the code of an image for which normal decoding can be performed by the above rearrangement process. A decoding process for decoding the encoded data of the image until the encoded data is transmitted. 5. The moving picture decoding method according to claim 4, wherein the decoding step interrupts the decoding processing when the coded data of the image for which the normal decoding processing cannot be performed is transmitted. Or not, on the basis of a user operation signal. 6. The moving picture decoding method according to claim 4, wherein the decoding step comprises: coded data of an image transmitted by the rearrangement processing;
A shape determining step of determining whether or not the image includes shape information indicating a shape of the image, wherein when the encoded data of the image includes shape information, the encoded data of the image in which the normal decoding process cannot be performed. Is transmitted, the decoding process for the coded data is suspended from the time when the coded data is transmitted until the coded data of the image that can be normally decoded is transmitted, If the encoded data of the image does not include the shape information, regardless of whether the encoded data of the image transmitted by the rearrangement process can perform the normal decoding process, A moving picture decoding method characterized by continuing decoding processing on the encoded data. 7. A moving picture decoding apparatus for decoding a coded stream obtained by coding image data of a moving picture composed of images for which individual times have been set, wherein the coded stream is provided as a coded stream from a transmission side. A data rearranging unit that receives encoded data of each image and outputs the received encoded data of each image in order from the earliest individual time set for each image at a transmission time corresponding to each image. A decoding unit that performs a decoding process on the encoded data of each image sent from the data rearranging unit to generate decoded image data corresponding to each image; and a decoding unit that is output from the decoding unit. A display image output unit that outputs decoded image data corresponding to each of the images as display image data so that display of each image is performed at a predetermined timing. When the encoded data of the image to be output is received later than the second transmission time, which is later than the first transmission time, the individual time is set to a time within the range from the first transmission time to the second transmission time. A moving image decoding apparatus for transmitting encoded data of an image. 8. The moving picture decoding apparatus according to claim 7, wherein the display image output unit is configured such that, in the data rearrangement unit, the individual time ranges from a first transmission time to a second transmission time. When the encoded data of the image which is the time within is transmitted,
A moving image decoding unit for outputting, as display image data, only decoded image data of an image whose individual time is closest to the second transmission time among the images to which the encoded data is transmitted. apparatus. 9. A moving picture decoding apparatus for decoding a coded stream obtained by coding image data of a moving picture composed of images for which individual times have been set, wherein the coded stream is provided as a coded stream from a transmission side. A data reordering unit that receives the encoded data of each image, and transmits the received encoded data of each image in ascending order of the individual time set for each image; and outputting from the data reordering unit. A decoding unit that performs a decoding process on the encoded data of each of the generated images to generate decoded image data corresponding to each image; and a decoded image data corresponding to each image output from the decoding unit. And a display image output unit that outputs the image as display image data so that the display of each image is performed at a predetermined timing. When the encoded data is transmitted, the encoding of the image is performed from the time when the encoded data is transmitted until the encoded data of the image that can be normally decoded is transmitted from the data rearranging unit. A moving picture decoding apparatus for interrupting decoding of data. 10. The video decoding device according to claim 9, wherein the display image output unit is configured to perform the decoding unit while the decoding unit suspends the decoding process on the encoded data of the image. A moving image decoding apparatus for outputting, as display image data, decoded image data that has been subjected to decoding processing last. 11. A program recording medium storing a program for performing, by a computer, a decoding process on an encoded stream obtained by encoding image data of a moving image composed of images each having an individual time set, The above program is such that the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image in ascending order of the individual time set for each image. Performs a rearrangement process of sending out the image at the transmission time corresponding to each image, and in this rearrangement process, the computer converts encoded data of the image to be sent out at the first transmission time to a second time later than the first transmission time. If the received time is later than the transmission time, the computer determines that the individual time is a time within the range from the first transmission time to the second transmission time. A rearrangement program for sending out the encoded data of the image, and the computer performs a decoding process on the encoded data of each image sent out by the above-mentioned rearrangement process, and decodes the decoded image data corresponding to each image. And a decryption program for generating the program recording medium. 12. A program storage medium storing a program for performing, by a computer, a decoding process on an encoded stream obtained by encoding image data of a moving image composed of images each having an individual time set therein, The above program is such that the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image in ascending order of the individual time set for each image. A rearrangement program for performing a rearrangement process to be sent, and a computer performing a decoding process on the encoded data of each image transmitted by the rearrangement process to generate decoded image data corresponding to each image However, at the time of this decoding processing, when encoded data of an image for which normal decoding processing is impossible is transmitted by the rearrangement processing, The computer suspends the decoding process on the coded data of the image from the time when the coded data is transmitted to the time when the coded data of the image that can be normally decoded is transmitted by the rearrangement process. And a decryption program for performing the decryption. 13. A moving image decoding method for decoding a coded stream obtained by coding image data of a moving image composed of images each having an individual time set, wherein the coded stream is provided as a coded stream from a transmission side. Performs a rearrangement process of receiving the encoded data of each image and sending out the received encoded data of each image to the transmission time corresponding to each image in order from the earliest individual time set for each image. A rearrangement step; and a decoding step of performing a decoding process on the encoded data of each image transmitted by the rearrangement process to generate decoded image data corresponding to each image. The step is a step of receiving encoded data of an image to be transmitted at the required transmission time if the encoded data of the image to be transmitted at the required transmission time cannot be received by the required transmission time. Transmission of the encoded data of the image is stopped until receiving the encoded data of the image to be transmitted at the required transmission time, and then the transmission of the encoded data of the image is restarted. Moving image decoding method. 14. A moving picture decoding method for decoding a coded stream obtained by coding image data of a moving picture composed of images for which individual times are set, wherein the coded stream is transmitted as a coded stream from a transmission side. Performs a rearrangement process of receiving the encoded data of each image and sending out the received encoded data of each image to the transmission time corresponding to each image in order from the earliest individual time set for each image. A rearrangement step; and a decoding step of performing a decoding process on the encoded data of each image transmitted by the rearrangement process to generate decoded image data corresponding to each image. The step is a step of, if encoded data of an image to be sent at the required sending time cannot be received by the required sending time, until a predetermined condition is satisfied, or The transmission of the encoded data of the image is stopped until the encoded data of the image to be transmitted is received, and the encoded data of the image to be transmitted is received after the predetermined condition is satisfied or at the required transmission time. After doing
A moving picture decoding method characterized by restarting transmission of encoded picture data. 15. The moving picture decoding method according to claim 14, wherein the predetermined condition is that the remaining storage capacity of the buffer memory for storing the received coded stream of each image is a predetermined value during the rearrangement process. A moving picture decoding apparatus characterized by the following. 16. The moving picture decoding method according to claim 14, wherein the predetermined condition is that, during the rearrangement processing, the time during which the transmission of the encoded data of the image is stopped is equal to or longer than a predetermined value. A moving picture decoding apparatus. 17. A moving picture decoding apparatus for decoding a coded stream obtained by coding image data of a moving picture composed of images for which individual times have been set, wherein the coded stream is provided as a coded stream from a transmission side. A data rearranging unit that receives encoded data of each image and outputs the received encoded data of each image in order from the earliest individual time set for each image at a transmission time corresponding to each image. A decoding unit that performs a decoding process on the encoded data of each image output from the data rearranging unit to generate decoded image data corresponding to each image; and a decoding unit that is output from the decoding unit. A display image output unit that outputs decoded image data corresponding to each image as display image data so that display of each image is performed at a predetermined timing. If the encoded data of the image to be transmitted cannot be received by the required transmission time, the transmission of the encoded data of the image is stopped until the encoded data of the image to be transmitted at the required transmission time is received, A moving picture decoding method characterized in that after receiving encoded data of an image to be transmitted at the required transmission time, transmission of the encoded data of the image is restarted. 18. A moving picture decoding apparatus for decoding a coded stream obtained by coding image data of a moving picture composed of images for which individual times have been set, wherein the coded stream is provided as a coded stream from a transmission side. A data rearranging unit that receives encoded data of each image and outputs the received encoded data of each image in order from the earliest individual time set for each image at a transmission time corresponding to each image. A decoding unit that performs a decoding process on the encoded data of each image output from the data rearranging unit to generate decoded image data corresponding to each image; and a decoding unit that is output from the decoding unit. A display image output unit that outputs decoded image data corresponding to each image as display image data so that display of each image is performed at a predetermined timing. If the encoded data of the image to be transmitted cannot be received by the required transmission time, the image data is transmitted until the predetermined condition is satisfied or the encoded data of the image to be transmitted at the required transmission time is received. After stopping the transmission of the encoded data, after the predetermined condition is satisfied, or after receiving the encoded data of the image to be transmitted at the required transmission time,
A moving image decoding apparatus for restarting transmission of encoded image data. 19. The moving picture decoding device according to claim 18, wherein the display image output unit is configured to stop transmitting the encoded data of the image to the decoding unit in the data rearranging unit. When the transmission of the encoded data of the image is restarted, the image display within a certain period is performed so that the delay of the display timing of the image caused by the stop of the transmission becomes smaller each time the image is displayed. A moving picture decoding apparatus characterized by the following. 20. A program storage medium storing a program for performing, by a computer, a decoding process on an encoded stream obtained by encoding image data of a moving image composed of images each having an individual time set, The above program is such that the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image in ascending order of the individual time set for each image. Performing a rearrangement process for sending out at the transmission time corresponding to each image, and in this rearrangement process, when the computer cannot receive the encoded data of the image to be transmitted at the required transmission time by the required transmission time. Until the computer receives the encoded data of the image to be transmitted at the required transmission time, the encoded data of the image is After stopping the output and receiving the encoded data of the image to be transmitted at the required transmission time, a rearrangement program for resuming the transmission of the encoded data of the image and the computer are transmitted by the rearrangement process. And a decoding program for performing a decoding process on the encoded data of each image to generate decoded image data corresponding to each image. 21. A program storage medium storing a program for performing, by a computer, a decoding process on an encoded stream obtained by encoding image data of a moving image composed of images each having an individual time set, The above program is such that the computer receives the encoded data of each image as an encoded stream from the transmission side, and converts the received encoded data of each image in ascending order of the individual time set for each image. Performing a rearrangement process for sending out at the transmission time corresponding to each image, and in the case of the rearrangement process, the computer cannot receive encoded data of an image to be transmitted at a required transmission time by the required transmission time. Is the encoded data of the image to be transmitted by the computer until a predetermined condition is satisfied or at the required transmission time. Until the reception, the transmission of the encoded data of the image is stopped, and after the predetermined condition is satisfied, or after the encoded data of the image to be transmitted at the required transmission time is received, the encoded data of the image is A rearranging program for resuming the transmission, and a computer for performing a decoding process on the encoded data of each image transmitted by the rearranging process to generate decoded image data corresponding to each image. And a decoding program for decoding a moving image.