JP2021052429A - Moving picture prediction decoding method - Google Patents

Abstract

To contribute to prediction performance improvement, by making a decodable picture available as a reference picture of a subsequent picture.SOLUTION: A moving picture prediction decoding method includes: an input step for inputting a bit stream including compression image data having information of an NAL unit type for identifying a picture as one of a plurality of picture types including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture, for each picture; a restoration step of decoding the compression image data, on the basis of the picture type, and restoring to a reproduction image; and an output step of outputting the reproduction image. In the restoration step, the moving picture prediction decoding method decodes the non-RAS reading picture and skips decoding of the RAS reading picture when the restoration step is started from the random access picture.SELECTED DRAWING: Figure 2

Description

The present invention relates to a moving image prediction decoding method, and more particularly to a moving image prediction decoding method relating to interscreen prediction effective for random access.

Compression coding technology is used to efficiently transmit and store moving image data. In the case of moving images, MPEG1-4 and H.M. 261-H. The 264 method is widely used.

In these coding methods, the image to be coded is divided into a plurality of blocks, and then the coding / decoding process is performed. The following predictive coding method is used to improve the coding efficiency. In the predictive coding in the screen, a predictive signal is generated using an adjacent already-reproduced image signal (reconstructed image data compressed in the past) in the same screen as the target block, and then the target is targeted. Encode the difference signal subtracted from the block signal. In predictive coding between screens, the already reproduced image signal in the screen different from the target block is referred to, the displacement of the signal is searched, the movement amount is compensated to generate the predictive signal, and the prediction signal is generated and used as the target block. The difference signal subtracted from the signal of is encoded. A replayed image that is referenced to search for and compensate for movement is called a reference image.

Further, in the bidirectional screen-to-screen prediction, not only the images in the past in the order of output time but also the future image output after the target image may be referred to (however, this future image is ahead of the target image). It is necessary to encode it into and play it in advance). By averaging the prediction signals obtained from the past image and the future image, it is effective in predicting the signal of the hidden and newly appearing object, and at the same time, the noise contained in both prediction signals is reduced. Has the effect of

In addition, H. In the inter-screen prediction coding of 264, the prediction signal for the target block refers to a plurality of reference images encoded and reproduced in the past, and the image signal having the smallest error is used as the optimum prediction signal while performing motion search. select. The difference between the pixel signal of the target block and this optimum prediction signal is obtained, subjected to discrete cosine transform, quantized, and then entropy-encoded. At the same time, information on which region of which reference image the optimum prediction signal for the target block is acquired (referred to as a reference index and a motion vector, respectively) is also encoded. H. In 264, the reproduced 4 to 5 images are stored in the frame memory or the reproduced image buffer (decoded picture buffer) as reference images.

Although screen-to-screen predictive coding can efficiently perform compression coding by taking advantage of the correlation between images, in order to enable viewing of a video program from the middle by switching TV channels, the dependency between screens is required. It needs to be cut off. In the compressed bitstream of a moving image, the part where there is no dependency between screens is called a random access point. In addition to switching channels, a random access point is also required for editing moving images and connecting compressed data of different moving images. A "clean random access point" is provided. A clean random access point is identified by a NAL unit type clean random access picture ("CRA picture"). A bitstream can contain a plurality of CRA pictures, and the moving image predictive decoding device may start decoding from any clean random access point.

For the disclosure of the invention, the picture types of pictures associated with CRA pictures are defined below (see FIG. 10).
a) Past picture: A picture that is decoded before the CRA picture and output before the CRA picture.
b) Lagging picture: A picture that is decoded before the CRA picture but output after the CRA picture
c) Reading picture: A picture that is decoded after the CRA picture but output before the CRA picture
d) Normal picture: A picture that is decoded after the .CRA picture and output after the CRA picture

Since the CRA picture is defined as a picture with only in-screen prediction, it has all the information necessary for decoding and can be correctly decoded without referring to other pictures. It is defined that all normal pictures following a CRA picture must not make interscreen predictions from past pictures, lagging pictures, or reading pictures.

When decoding a bitstream from a CRA picture, it is guaranteed that the CRA picture and the normal picture are correctly decoded without any interscreen prediction errors. However, the reading picture is decoded after the CRA picture, and there is no guarantee that it will be decoded correctly without error in interscreen prediction. That is, while there may be a reading picture that is correctly decoded, there may be a reading picture that is not correctly decoded.

Note that "correctly decoded" here means that the decoded picture is the same as the picture obtained when the decoded picture is decoded from the beginning of the bitstream, not from the CRA picture. When decoding from a CRA picture, the picture (for example, a lagging picture) preceding the CRA picture in the decoding order is not decoded and does not exist in the reproduced image buffer. Therefore, a subsequent picture that directly or indirectly uses a picture preceding the CRA picture in the decoding order for inter-screen prediction may contain a decoding error.

Benjamin Bross et. Al., "High efficiency video coding (HEVC) text specification draft 7", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 9th Meeting : Geneva, CH, 27 April --7 May 2012

As described above, when the moving image prediction decoding device starts decoding from a random access point, there may be a picture that cannot be correctly decoded, and the picture that cannot be correctly decoded cannot be used for subsequent decoding. .. On the other hand, some pictures can be correctly decoded, and the correctly decoding pictures can be used for subsequent decoding. In the prior art, since there is no way to specify which picture following the random access point is discarded in the decoding order, all the reading pictures are treated as pictures that cannot be correctly decoded and discarded. However, some of these pictures are actually decodable and can contribute to improving the predictive performance of subsequent pictures, so a method of discarding all of the reading pictures as pictures that cannot be correctly decoded is a desirable method. is not it.

Therefore, an object of the present invention is to make it possible to discriminate a decodable picture so that the decodable picture can be used as a reference picture of a subsequent picture and contribute to improvement of prediction performance.

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, which is compressed image data for a plurality of pictures constituting a moving image, and is for each picture. , Has NAL unit type information that identifies the picture as one of a plurality of picture types including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture. An input step for inputting a bit stream including compressed image data, a restoration step for decoding the compressed image data based on the picture type and restoring the reproduced image, and an output step for outputting the reproduced image. Equipped with
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
The reference picture set of the non-RAS reading picture, including the reference picture used for interscreen prediction of the non-RAS reading picture, precedes the RAS reading picture and the associated random access picture in decoding order. If the second random access picture is decoded after the first random access picture without including any of the pictures, then the reference picture set of the second random access picture is the first random access picture. If the random access picture does not include any of the pictures that precede it in the decoding order and the decoding process is started from the random access picture in the restore step, then the non-RAS reading picture is decoded and the RAS reading picture is decoded. It is characterized by skipping.

According to the present invention, by making it possible to discriminate a decodable picture, the decodable picture can be used as a reference picture of a subsequent picture, which can contribute to the improvement of prediction performance. More specifically, whether the moving image predictive decoding device can correctly decode a certain picture (by using a label or collating with a reference picture set) when decoding is started from the CRA picture at the beginning of the bitstream. Can be detected. Therefore, instead of discarding all the reading pictures, the moving image prediction decoding device can select and discard only the pictures that cannot be decoded, and the decodable pictures can be used as the reference pictures of the subsequent pictures. , Can contribute to the improvement of prediction performance.

It is a block diagram which shows the moving image prediction coding apparatus which concerns on embodiment of this invention. It is a block diagram which shows the moving image prediction decoding apparatus which concerns on embodiment of this invention. It is a figure for demonstrating the syntax element which concerns on embodiment of this invention. It is a flow chart which shows the moving image prediction coding method which concerns on embodiment of this invention. It is a flow chart which shows the moving image prediction decoding method which concerns on embodiment of this invention. It is a figure which shows the hardware configuration of the computer for executing the program recorded on the recording medium. It is an overview diagram of a computer for executing a program stored in a recording medium. It is a block diagram which shows the structural example of the moving image prediction coding program. It is a block diagram which shows the structural example of the moving image prediction decoding program. It is a figure for demonstrating the background of this invention.

Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1 to 9.

[About moving image prediction coding device]
FIG. 1 is a functional block diagram showing the configuration of the moving image prediction coding device 100 according to the present embodiment. As shown in FIG. 1, the moving image prediction coding device 100 has an input terminal 101, a block divider 102, a prediction signal generator 103, a frame memory 104, a subtractor 105, a converter 106, and a quantum as functional configurations. It includes a converter 107, an inverse quantizer 108, an inverse converter 109, an adder 110, an entropy encoder 111, an output terminal 112, an input terminal 113, and a frame memory manager (also referred to as a buffer manager) 114. The operation of each functional block will be described in the operation of the moving image prediction coding device 100 described later. The converter 106 and the quantizer 107 correspond to the coding means, and the inverse quantizer 108, the inverse converter 109 and the adder 110 correspond to the decoding means.

The operation of the moving image prediction coding device 100 configured as described above will be described below. A moving image signal composed of a plurality of images is input to the input terminal 101. The image to be coded is divided into a plurality of regions by the block divider 102. In the present embodiment, the block is divided into blocks composed of 8 × 8 pixels, but the block may be divided into other block sizes or shapes. Next, a prediction signal is generated for a region (hereinafter referred to as “target block”) to be coded. In this embodiment, two types of prediction methods are used. That is, inter-screen prediction and in-screen prediction.

In the inter-screen prediction, a reproduced image that has been encoded in the past and then restored is used as a reference image, and motion information that gives a prediction signal having the smallest error with respect to the target block is obtained from this reference image. This process is called motion detection. Further, depending on the case, the target block may be subdivided, and the inter-screen prediction method may be determined for the subdivided small area. In this case, the most efficient division method and the motion information of each are determined from the various division methods for the entire target block. In the embodiment according to the present invention, it is performed by the prediction signal generator 103, the target block is input via the line L102, and the reference image is input via L104. As the reference image, a plurality of images encoded and restored in the past are used as the reference image. For details, see MPEG-2, 4, H.M. It is the same as any method of 264. The motion information determined in this way and the method of dividing the small area are sent to the entropy encoder 111 via the line L112, encoded, and then sent out from the output terminal 112. In addition, information (reference index) regarding which reference image the prediction signal is acquired from among the plurality of reference images is also sent to the entropy encoder 111 via the line L112. In the embodiment according to the present invention, four to five reproduced images are stored in the frame memory 104 and used as reference images. The prediction signal generator 103 acquires a reference image signal from the frame memory 104 based on the reference image and motion information corresponding to the method of dividing the small area and each small area, and generates the prediction signal. The interscreen prediction signal generated in this way is sent to the subtractor 105 via the line L103.

In the in-screen prediction, an in-screen prediction signal is generated using the already reproduced pixel values that are spatially adjacent to the target block. Specifically, the prediction signal generator 103 acquires an already reproduced pixel signal in the same screen from the frame memory 104, and extrapolates these signals to generate an in-screen prediction signal. Information on the extrapolation method is sent to the entropy encoder 111 via the line L112, encoded, and then sent from the output terminal 112. The in-screen prediction signal generated in this way is sent to the subtractor 105. The in-screen prediction signal generation method in the prediction signal generator 103 is a conventional technique, H.I. It is the same as the method of 264. With respect to the inter-screen prediction signal and the in-screen prediction signal obtained as described above, the one having the smallest error is selected and sent to the subtractor 105.

As for the first image, since there is no image before that, all the target blocks are processed by the in-screen prediction. In addition, the target block, which is periodically set as a random access point, is processed by in-screen prediction in preparation for switching TV channels. Such an image is called an intraframe, and H.I. In 264, it is called an IDR picture.

The subtractor 105 subtracts the prediction signal (via line L103) from the signal of the target block (via line L102) to generate a residual signal. The residual signal is discrete cosine transformed by the converter 106, and each coefficient is quantized by the quantizer 107. Finally, the conversion coefficient quantized by the entropy encoder 111 is encoded and sent from the output terminal 112 together with information on the prediction method.

The compressed signal of the target block is reverse-processed and restored in order to perform in-screen prediction or inter-screen prediction for the subsequent target block. That is, the quantized conversion coefficient is inversely quantized by the inverse quantizer 108 and then inverse discrete cosine transform is performed by the inverse converter 109 to restore the residual signal. The residual signal restored by the adder 110 and the prediction signal sent from the line L103 are added, the signal of the target block is reproduced, and the signal is stored in the frame memory 104. Although the converter 106 and the inverse converter 109 are used in this embodiment, other conversion processes may be used instead of these converters. In some cases, the converter 106 and the inverse converter 109 may not be present.

The frame memory 104 is finite, and it is impossible to store all the reproduced images. Only the reproduced image used for encoding the subsequent image is stored in the frame memory 104. The frame memory manager 114 controls the frame memory 104. The frame memory manager 114 selects unnecessary (for example, the oldest) reproduced images of N images (here, N = 4, but may be a predetermined integer) in the frame memory 104. It is controlled via the line L115 so that the most recently reproduced image to be erased and used as a reference image can be stored. The output order information of each image, the type for encoding the image (in-screen predictive coding, inter-screen predictive coding, bidirectional predictive coding), and the reference index are also input from the input terminal 113 via the line L112. The frame memory manager 114 operates based on the information in.

At the same time, the output order information of each image and the NAL unit type information described later are sent to the entropy encoder 111 via the line L114 as needed, and are output together with the encoded and compressed image data. .. The output order information is attached to each image, and may be an image indicating the order of the images, the time when the images are output, or the output reference time (temporal reference) of the images. In the present embodiment, the value of the output order information is directly converted into binary coding. The operation of the frame memory manager 114 of this embodiment will be described later.

[About moving image prediction decoding device]
Next, the moving image prediction decoding device according to the present invention will be described. FIG. 2 is a functional block diagram showing the configuration of the moving image prediction / decoding device 200 according to the present embodiment. As shown in FIG. 2, the moving image prediction / decoding device 200 has an input terminal 201, a data analyzer 202, an inverse quantizer 203, an inverse converter 204, an adder 205, and a prediction signal generator 208 as functional configurations. , Frame memory 207, output terminal 206, frame memory manager 209, controller 210, and switch 211. The operation of each functional block will be described in the operation of the moving image prediction / decoding device 200 described later. The inverse quantizer 203 and the inverse converter 204 correspond to the decoding means. The decoding means is not limited to the inverse quantizer 203 and the inverse converter 204, and other means may be used. Further, the means for decoding may be configured by eliminating the inverse converter 204 and using only the inverse quantizer 203.

Hereinafter, the operation of the moving image prediction decoding device 200 will be described. The compressed data compressed and encoded by the method described above by the moving image prediction coding device 100 is input from the input terminal 201. This compressed data includes information related to the generation of the predicted and encoded residual signal and the predicted signal for the target block in which the image is divided into a plurality of blocks. Information related to the generation of the prediction signal includes information on block division (block size) in the case of inter-screen prediction, motion information and information on the above-mentioned reference index and NAL unit type, and is used for in-screen prediction. In the case, information on the method of extrapolation from the peripheral already reproduced pixels is included.

The data analyzer 202 extracts the residual signal of the target block, the information related to the generation of the prediction signal, the quantization parameter, and the output order information of the image from the compressed data. The residual signal of the target block is inversely quantized by the inverse quantizer 203 based on the quantization parameters (via lines L202 and L211). The result is inverse discrete cosine transformed by inverse transformer 204.

Information related to the generation of the predicted signal is then sent to the predicted signal generator 208 via line L206b. The prediction signal generator 208 accesses the frame memory 207 based on the information related to the generation of the prediction signal, acquires the reference signal from the plurality of reference images (via the line L207), and generates the prediction signal. This prediction signal is sent to the adder 205 via the line L208, is added to the restored residual signal, reproduces the target block signal, is output from the output terminal 206 via the line L205, and is stored in the frame memory 207 at the same time. ..

The frame memory 207 stores a reproduced image used for decoding / reproducing a subsequent image. The frame memory manager 209 controls the frame memory 207 via the line L209a. The frame memory 207 erases unnecessary images (for example, the oldest one) from the stored N images (here, N = 4, but may be a predetermined integer) and is a reference image. It is controlled so that the most recently reproduced image used as can be stored.

The controller 210 operates based on the output order information of the target image sent via the line L206a and the information regarding the coding type and the NAL unit type of the image. Alternatively, it operates based on the reference index sent via the line L206a and the information of the frame stored in the frame memory sent via the line L209b. The operation of the controller 210 according to the present invention will be described later.

The switch 211 is controlled from the controller 210 via the line L210, and operates so as to skip decoding of a specific frame depending on the conditions. The operation of the switch 211 according to the present invention will be described later.

FIG. 3 shows the bitstream syntax element 500. The bitstream syntax element 500 is composed of a plurality of syntax elements necessary for decoding one picture (510, 520, etc.). In the syntax of the picture, we pay attention to the following three elements.
1) Network adaptation layer unit type (NUT) or NAL unit type (530)
2) Picture Output Count (POC) (540)
3) Reference Picture Set (RPS) (550)

1) The NUT contains information about the picture type. Note that other means of signaling the picture type can also be used in the present invention. In this embodiment, each picture is labeled as one of three NAL unit types. The NAL unit type is RAS, CRA, non-RAS described below.

A picture labeled as a RAS (Random Access Skip) picture is skipped and not output when decoding is initiated from the CRA picture associated with that RAS picture. On the other hand, if the CRA picture is not the first picture of the bit storm (or decoding is not started from the CRA picture), the moving image predictive decoding device 200 considers the RAS picture to be a non-RAS picture. , Decode and output according to the output instruction of the picture.

A picture labeled as a CRA (Clean Random Access) picture correctly decodes a non-RAS picture associated with the CRA picture when the decoding of the bitstream is started from that CRA picture. It shows that it can be done.

It is assumed that the picture labeled as a non-RAS picture is decoded by the moving image prediction decoding apparatus 200 and output according to the output instruction of the picture. CRA pictures are considered non-RAS pictures unless otherwise noted.

2) The POC contains information on the order of the output pictures.

3) The RPS contains information on the reference picture used for interscreen prediction of the current picture. A reference picture in the reproduction image buffer (DPB) that does not exist in the RPS cannot be used as a reference picture for predictive decoding by the current picture or any of the pictures.

In the present embodiment, in order to ensure that all non-RAS pictures can be correctly decoded when the decoding of the bitstream is started from the CRA picture, the RPS has the following features.
Feature 1: With respect to the RPS utilized by the reading picture, if one or more reference pictures (or at least one reference picture) are RAS pictures or precede the associated CRA picture in decoding order, the reading. The picture is a RAS picture.
Feature 2: All reference pictures in the RPS used by the non-RAS picture are reference pictures that are non-RAS pictures and reference pictures that are decoded after the associated CRA picture.

In this embodiment, since the normal picture is treated as a non-RAS picture, a picture that does not satisfy features 1 and 2 is not allowed in the bitstream. However, the present invention is not limited to the reading picture described in Feature 1, and can be applied equally to all pictures. With respect to feature 2, the present invention can also be applied when the reference picture is limited to the reading picture only.

[Characteristic operation in moving image prediction coding device 100]
The operation of the moving image prediction coding device 100 for generating the bit stream having the above-mentioned characteristics, which is the point of the present invention, will be described with reference to FIG. The moving image prediction coding device 100 inserts CRA pictures at regular intervals in order to realize random access. All the pictures following the inserted CRA picture in the coding order are associated with the inserted CRA picture until the next CRA picture is inserted, and are encoded according to the following steps.

In step 620, it is determined whether or not one or more of the reference pictures included in the RPS of the picture is a RAS picture. If one or more of the reference pictures included in the RPS of the picture are RAS pictures (YES), the process proceeds to step 650, and if not (NO), the process proceeds to step 630.

In step 630, it is determined whether or not one or more of the reference pictures included in the RPS of the picture precedes the CRA picture associated with the picture in the coding order. If one or more of the reference pictures included in the RPS of the picture precedes the CRA picture associated with the picture in the coding order (YES), the process proceeds to step 650, and if not (NO), the process proceeds to step 640. ..

In step 650, the POC of the picture is compared with the POC of the CRA picture associated with the picture to check if the picture is a leading picture. If the POC of the picture is smaller than the POC of the CRA picture associated with the picture, it is determined that the picture is a leading picture (YES) and the process proceeds to step 670. If not, it is determined that the picture is not a reading picture (NO), but the determinations in steps 620 and 630 are only for the reading picture (YES), and the picture is not a reading picture (NO). ) Is abnormal, so the process proceeds to step 660, an error message is output, and the process proceeds to step 680. After outputting the error message in step 660, the process of FIG. 4 may be terminated abnormally.

In step 670, the picture is encoded as a RAS picture, and the information that the picture is a RAS picture (NAL unit type: RAS) is encoded. Then proceed to step 680.

In step 640, the picture is encoded as a non-RAS picture, and the information that the picture is a non-RAS picture (NAL unit type: non-RAS) is encoded. Then proceed to step 680. Here, unless otherwise specified, the CRA picture is included in the non-RAS picture.

In steps 640 and 670, the information about whether the picture is a RAS picture or a non-RAS picture does not necessarily have to be encoded, and instead of encoding the information, the reference picture list and frame memory of each picture. It may be determined whether the picture is a RAS picture or a non-RAS picture by collating with the picture stored in 104.

In step 680, the moving image prediction coding device 100 determines whether or not there are more pictures to be encoded, and if there is (YES), the process returns to step 620 and the process is repeated, while if there is not (NO), the figure is shown. The process of 4 is completed.

The series of processes described above corresponds to the processes of the entire moving image prediction coding device 100 of FIG. 1, and in particular, the determination processes of steps 620, 630, and 650 are performed by the frame memory manager 114.

[Characteristic operation in moving image prediction / decoding device 200]
In the moving image prediction decoding apparatus 200 according to the present embodiment, the operation differs depending on whether the decoding process is started from the CRA picture as the first picture of the bitstream and the case where the first picture of the bitstream is not the CRA picture. This decoding process returns to the normal decoding process when the next CRA picture is decoded.

The operation of the moving image prediction decoding apparatus 200 for decoding the bit stream having the above-mentioned characteristics, which is the point of the present invention, will be described with reference to FIG.

In step 710, the moving image predictive decoding apparatus 200 determines whether or not the first picture of the bitstream (that is, the first picture that starts decoding the bitstream) is a CRA picture based on the NAL unit type. .. If the first picture is not a CRA picture (NO), the process proceeds to step 780, and the moving image prediction decoding apparatus 200 decodes each picture as usual. That is, in this step 780, the RAS picture is regarded as a non-RAS picture, and is decoded and output according to the instructions in the picture as usual. On the other hand, if the first picture of the bitstream is a CRA picture in step 710 (YES), the process proceeds to step 720.

The process from step 720 to step 770 is repeatedly executed for all the pictures during the period until immediately before the next CRA picture starts to be decoded, and then returns to the normal decoding process in step 780. Hereinafter, the processes from step 720 to step 770 will be described.

In step 720, the moving image prediction decoding apparatus 200 determines whether or not the picture is correctly decoded at the start of decoding the picture. Since the bitstream in the present embodiment has the above-mentioned features 1 and 2, the moving image prediction decoding apparatus 200 uses at least one of the following two methods to determine whether or not the picture can be correctly decoded. It can be determined. The first method is to look at the NAL unit type label of the picture. If the picture is labeled as a RAS picture, it can be determined that the picture cannot be decoded correctly. The second method is that the moving image prediction / decoding device 200 collates the reference picture existing in the DPB with the reference picture list of the RPS of the picture. If any of the reference pictures in the RPS of the picture does not exist in the DPB, it can be determined that the picture cannot be decoded correctly. If the moving image prediction decoding apparatus 200 determines that the picture can be correctly decoded by using at least one of the first and second methods as described above (YES), the process proceeds to step 730, and the picture is correctly decoded. If it is determined that the decryption cannot be performed (NO), the process proceeds to step 750.

In step 730, the moving image prediction decoding apparatus 200 decodes and outputs the picture according to the instructions in the picture. This also applies to CRA pictures. Then, the process proceeds to step 740.

In step 750, the POC of the picture is compared with the POC of the CRA picture associated with the picture to determine if the picture is a leading picture. If the POC of the picture is smaller than the POC of the CRA picture associated with the picture (YES), the picture is determined to be a leading picture and the process proceeds to step 770 described below. If this is not the case (NO), the picture is not a reading picture and an error occurs. Therefore, the process proceeds to step 760, and the moving image prediction / decoding device 200 outputs an error message and proceeds to step 740. After outputting the error message in step 760, the process of FIG. 5 may be terminated abnormally. Also, as mentioned above, it should be noted that the determination in step 750 is necessary only when feature 1 is limited to the reading picture.

In step 770, the moving image prediction decoding device 200 skips decoding the picture and performs the following necessary housekeeping processing. Here, examples of the necessary housekeeping process include a process of labeling the skipped picture to indicate that it cannot be used as a reference picture and is not output. Then, the process proceeds to step 740.

In step 740, the moving image prediction decoding apparatus 200 determines whether or not the next picture to be decoded is a CRA picture, and if the next picture is not a CRA picture (NO), the process returns to step 720 and the process is repeated. .. On the other hand, when the next picture is a CRA picture (YES), the decoding process (random access decoding process) according to the present invention is not required after the next CRA picture. All pictures are decoded and output according to the output instruction information).

The series of processes described above corresponds to the processes of the entire moving image prediction / decoding device 200 of FIG. 2, and in particular, the determination of steps 720 and 750 and the control of steps 730 and 770 are performed by the controller 210.

According to the present embodiment as described above, when decoding is started from the CRA picture at the beginning of the bit stream, the moving image prediction decoding device 200 (by using a label or collating with a reference picture set). It is possible to detect whether or not a picture can be correctly decoded. Therefore, the moving image prediction decoding device 200 can select and discard only the pictures that cannot be decoded instead of discarding all the reading pictures, and the decodable picture can be used as a reference picture of the subsequent picture. It can contribute to the improvement of prediction performance.

When the NAL unit type RAS is assigned to the picture, the moving image prediction coding device 100 generates a picture that can be correctly decoded and a picture that cannot be correctly decoded. On the other hand, the moving image prediction decoding device 200 does not output a picture that cannot be correctly decoded. This creates a time gap in the output picture, which may affect the output rate of the frame. In some systems, gaps in output are not desirable. In the present embodiment, the moving image prediction coding device 100 additionally determines whether or not there is a gap in these RAS pictures in the CRA picture header or the video usability syntax (YUI). The moving image prediction decoding device 200 is instructed by the flag of. Upon receiving this flag, the moving image prediction decoding device 200 can select whether or not to output a reading picture that can be correctly decoded but has a gap.

Further, as another means different from the above, a further restriction may be provided on the bit stream so that a gap does not occur in the RAS picture preceding the CRA picture in the output order. That is, the bit stream may be continuously output so as not to generate a gap in the RAS picture.

As yet another means, the moving image prediction decoding device 200 decodes the non-RAS reading picture without depending on other additional information from the moving image prediction coding device 100 or the output instruction information of the picture. May decide not to output.

In this embodiment, the NAL unit type (RAS, CRA, non-RAS) label is detected and used by the moving image predictive decoding device 200, while the NAL unit type label is other in the network. The device (eg, server, suitable network elements, etc.) may also be detected and used to perform the process of destroying the RAS picture when starting decoding from a random access point. This can save network bandwidth.

In this embodiment, each bitstream can contain a large number of CRA pictures, and there is a RAS picture associated with each CRA picture. If the second CRA picture follows the first CRA picture in the decoding order, it is not permissible for the RPS of the second CRA picture described above to contain a reference picture that is decoded before the first CRA picture. .. This guarantees that if the first CRA picture is the first picture in the bitstream, the RAS picture of the second CRA picture will be decoded.

[About moving image prediction coding program and moving image prediction decoding program]
The invention relating to the moving image predictive coding device 100 can be regarded as the invention relating to the moving image predictive coding program for causing the computer to function as the moving image predictive coding device 100. Similarly, the invention relating to the moving image prediction decoding device 200 can be regarded as the invention relating to the moving image prediction decoding program for causing the computer to function as the moving image prediction decoding device 200.

The moving image prediction coding program and the moving image prediction decoding program are provided, for example, stored in a recording medium. Examples of the recording medium include flexible disks, CD-ROMs, USB memories, DVDs, semiconductor memories, and the like.

FIG. 8 shows a module of a moving image predictive coding program for making a computer function as a moving image predictive coding device 100. As shown in FIG. 8, the moving image prediction coding program P100 includes an input module P101, a coding module P102, a restoration module P103, an image storage module P104, and a control module P105.

Further, FIG. 9 shows a module of a moving image prediction / decoding program for causing the computer to function as the moving image prediction / decoding device 200. As shown in FIG. 9, the moving image prediction / decoding program P200 includes an input module P201, a restoration module P202, an image storage module P203, and a control module P204.

The moving image prediction coding program P100 and the moving image prediction decoding program P200 configured as described above can be stored in the recording medium 10 shown in FIGS. 6 and 7, and are executed by the computer 30 described later.

FIG. 6 is a diagram showing a hardware configuration of a computer for executing a program recorded on a recording medium, and FIG. 7 is an overview diagram of a computer for executing a program stored on the recording medium. Computers also include DVD players, set-top boxes, mobile phones, etc. that have a CPU and perform processing and control by software.

As shown in FIG. 6, the computer 30 includes a reading device 12 such as a flexible disk drive device, a CD-ROM drive device, and a DVD drive device, a working memory (RAM) 14 in which an operating system is resident, and a recording medium 10. A memory 16 for storing a program stored in a computer, a display device 18 such as a display, a mouse 20 and a keyboard 22 as input devices, a communication device 24 for transmitting and receiving data and the like, and a CPU 26 for controlling program execution. And have. When the recording medium 10 is inserted into the reading device 12, the computer 30 can access the moving image predictive coding program stored in the recording medium 10 from the reading device 12 and executes the moving image predictive coding program. This makes it possible to operate as a moving image prediction coding device according to the present invention. Similarly, when the recording medium 10 is inserted into the reading device 12, the computer 30 becomes accessible from the reading device 12 to the moving image prediction / decoding program stored in the recording medium 10, and executes the moving image prediction / decoding program. By doing so, it becomes possible to operate as a moving image prediction / decoding device according to the present invention.

As shown in FIG. 7, the moving image prediction coding program or the moving image prediction decoding program may be provided via a network as a computer data signal 40 superimposed on a carrier wave. In this case, the computer 30 stores the moving image prediction coding program or the moving image prediction decoding program received by the communication device 24 in the memory 16 and executes the moving image prediction coding program or the moving image prediction decoding program. be able to.

The moving image prediction coding device according to the present invention is random by using an input means for inputting a plurality of images constituting a moving image and encoding the image by either in-screen prediction or inter-screen prediction. -The compressed image data including the image serving as the access point is generated, and the coding means for encoding the output order information of each image and the information regarding the picture type of each image and the compressed image data are decoded. A restoration means for restoring a reproduced image, an image storage means for storing one or more of the reproduced images as a reference image used for encoding a subsequent image, and the picture type are determined, and based on the determination result. The control means is provided with a control means for controlling the image storage means.
1) CRA picture: A picture characterized in that a type 2 picture following a CRA picture can be correctly decoded when decoding is started from the CRA picture.
2) Type 1 picture: A picture that is decoded after the CRA picture associated with the picture and output before the CRA picture, and is labeled as a type 1 picture for interscreen prediction. A picture having a list of reference pictures that includes at least one reference picture, or at least one reference picture that precedes the associated CRA picture in decoding order.
3) Type 2 picture: A picture having a list of reference pictures for interscreen prediction, and all the reference pictures in the list of reference pictures are labeled as type 2 pictures or CRA pictures. The picture, characterized in that it is decoded after the CRA picture associated with the picture.
As one of the three types in total, the image is labeled.

The moving image prediction coding method according to the present invention is a moving image prediction coding method executed by a moving image prediction coding device, in which an input step for inputting a plurality of images constituting a moving image and the image are combined. By encoding by either in-screen prediction or inter-screen prediction, compressed image data including images that serve as random access points is generated, and the output order information of each image and the picture type of each image are generated. A coding step for encoding information about, a restoration step for decoding the compressed image data and restoring it to a reproduced image, and one of the reproduced images as a reference image used for encoding a subsequent image. The image storage step for storing the above image and a control step for determining the picture type and controlling the image storage step based on the determination result are provided. In the control step, the moving image prediction coding device is provided.
1) CRA picture: A picture characterized in that a type 2 picture following a CRA picture can be correctly decoded when decoding is started from the CRA picture.
2) Type 1 picture: A picture that is decoded after the CRA picture associated with the picture and output before the CRA picture, and is labeled as a type 1 picture for interscreen prediction. A picture having a list of reference pictures that includes at least one reference picture, or at least one reference picture that precedes the associated CRA picture in decoding order.
3) Type 2 picture: A picture having a list of reference pictures for interscreen prediction, and all the reference pictures in the list of reference pictures are labeled as type 2 pictures or CRA pictures. The picture, characterized in that it is decoded after the CRA picture associated with the picture.
As one of the three types in total, the image is labeled.

The moving image prediction coding program according to the present invention encodes a computer by an input means for inputting a plurality of images constituting a moving image and the image by either in-screen prediction or inter-screen prediction. In addition to generating compressed image data including an image serving as a random access point, the encoding means for encoding the output order information of each image and the information regarding the picture type of each image, and the compressed image data are used. The picture type is determined and determined, the restoration means for decoding and restoring to the reproduced image, the image storage means for storing one or more of the reproduced images as a reference image used for encoding the subsequent image, and the picture type. The image storage means is made to function as a control means for controlling the image storage means based on the result, and the control means is used.
1) CRA picture: A picture characterized in that a type 2 picture following a CRA picture can be correctly decoded when decoding is started from the CRA picture.
2) Type 1 picture: A picture that is decoded after the CRA picture associated with the picture and output before the CRA picture, and is labeled as a type 1 picture for interscreen prediction. A picture having a list of reference pictures that includes at least one reference picture, or at least one reference picture that precedes the associated CRA picture in decoding order.
3) Type 2 picture: A picture having a list of reference pictures for interscreen prediction, and all the reference pictures in the list of reference pictures are labeled as type 2 pictures or CRA pictures. The picture, characterized in that it is decoded after the CRA picture associated with the picture.
As one of the three types in total, the image is labeled.

The moving image prediction / decoding device according to the present invention is a compression including a random access image obtained by encoding a plurality of images constituting a moving image by either in-screen prediction or inter-screen prediction. An input means for inputting image data, an output order of each image, and coded data indicating a picture type of each image, decoding the compressed image data and the coded data, and reproducing a reproduced image and output order information. Restoration means for restoring to picture type information, image storage means for storing one or more of the reproduced images as reference images used for decoding subsequent images, and the restoration based on the picture type. The picture type comprises a control means for controlling the means.
1) CRA picture: A picture characterized in that a type 2 picture following a CRA picture can be correctly decoded when decoding is started from the CRA picture.
2) Type 1 picture: A picture that is decoded after the CRA picture associated with the picture and output before the CRA picture, and is labeled as a type 1 picture for interscreen prediction. A picture having a list of reference pictures that includes at least one reference picture, or at least one reference picture that precedes the associated CRA picture in decoding order.
3) Type 2 picture: A picture having a list of reference pictures for interscreen prediction, and all the reference pictures in the list of reference pictures are labeled as type 2 pictures or CRA pictures. The picture, characterized in that it is decoded after the CRA picture associated with the picture.
Each image is labeled as one of a total of three types, and the restoring means decodes the image labeled as a type 2 picture when the decoding of the encoded data is started from the CRA picture. It is characterized in that a decoding process such as skipping the decoding of an image labeled as a type 1 picture is continued for a period until immediately before the processing of the next CRA picture.

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, and is based on either in-screen prediction or inter-screen prediction for a plurality of images constituting the moving image. An input step for inputting compressed image data including a random access image obtained by encoding, and coded data indicating the output order of each image and the picture type of each image, and the compressed image. A restoration step of decoding the data and the encoded data and restoring the reproduced image, output order information, and picture type information, and the reproduced image as a reference image used for decoding a subsequent image. The image storage step for storing the above image and a control step for controlling the restoration step based on the picture type are provided, and the picture type includes the image storage step.
1) CRA picture: A picture characterized in that a type 2 picture following a CRA picture can be correctly decoded when decoding is started from the CRA picture.
2) Type 1 picture: A picture that is decoded after the CRA picture associated with the picture and output before the CRA picture, and is labeled as a type 1 picture for interscreen prediction. A picture having a list of reference pictures that includes at least one reference picture, or at least one reference picture that precedes the associated CRA picture in decoding order.
3) Type 2 picture: A picture having a list of reference pictures for interscreen prediction, and all the reference pictures in the list of reference pictures are labeled as type 2 pictures or CRA pictures. The picture, characterized in that it is decoded after the CRA picture associated with the picture.
Each image is labeled as one of a total of three types, and in the restoration step, the moving image predictive decoding device labels the image as a type 2 picture when decoding of the encoded data is started from the CRA picture. It is characterized in that the decoding process of decoding the attached image and skipping the decoding of the image labeled as the type 1 picture is continued for a period until immediately before the processing of the next CRA picture.

The moving image prediction / decoding program according to the present invention is a random access image obtained by encoding a plurality of images constituting a moving image by either in-screen prediction or inter-screen prediction. An input means for inputting compressed image data including the image, output order of each image, and coded data indicating a picture type of each image, decoding the compressed image data and the coded data, and outputting a reproduced image and output. Based on the restoration means for restoring the sequence information and the picture type information, the image storage means for storing one or more of the reproduced images as reference images used for decoding the subsequent image, and the picture type. , The picture type, which functions as a control means for controlling the restoration means.
1) CRA picture: A picture characterized in that a type 2 picture following a CRA picture can be correctly decoded when decoding is started from the CRA picture.
2) Type 1 picture: A picture that is decoded after the CRA picture associated with the picture and output before the CRA picture, and is labeled as a type 1 picture for interscreen prediction. A picture having a list of reference pictures that includes at least one reference picture, or at least one reference picture that precedes the associated CRA picture in decoding order.
3) Type 2 picture: A picture having a list of reference pictures for interscreen prediction, and all the reference pictures in the list of reference pictures are labeled as type 2 pictures or CRA pictures. The picture, characterized in that it is decoded after the CRA picture associated with the picture.
Each image is labeled as one of a total of three types, and the restoring means decodes the image labeled as a type 2 picture when the decoding of the encoded data is started from the CRA picture. It is characterized in that a decoding process such as skipping the decoding of an image labeled as a type 1 picture is continued for a period until immediately before the processing of the next CRA picture.

By the way, the moving image prediction coding device, the method and the program, and the moving image prediction and decoding device, the method and the program according to the present invention can also adopt the following aspects.

The moving image prediction coding device according to the present invention is random by using an input means for inputting a plurality of images constituting a moving image and encoding the image by either in-screen prediction or inter-screen prediction. A coding means for generating compressed image data including an image serving as an access point and encoding output order information of each image, a restoring means for decoding the compressed image data and restoring it to a reproduced image, and the above. The control means includes an image storage means for storing one or more reproduced images as a reference image used for encoding a subsequent image, and a control means for controlling the image storage means.
1) CRA picture at which decoding of coded data is started,
2) A picture that is decoded after the CRA picture associated with the picture, output before the associated CRA picture, the decoding process by the restoration means is skipped, and the picture is not stored in the image storage means and is not output. The inter-screen prediction includes at least one reference picture in which the decoding process by the restoration means is skipped, or at least one reference picture that precedes the associated CRA picture in the decoding order. The picture, which has a list of referenced pictures,
3) A picture that is decoded by the restoration means and stored in the image storage means for reference as needed, and the picture has a list of reference pictures for performing interscreen prediction, and the reference picture. The picture, characterized in that all the reference pictures in the list are decoded by the restoring means and are decoded after the CRA picture associated with the picture.
The image is classified and controlled in a total of three.

The moving image prediction coding method according to the present invention is a moving image prediction coding method executed by a moving image prediction coding device, in which an input step for inputting a plurality of images constituting a moving image and the image are combined. A coding step that generates compressed image data including an image that serves as a random access point by encoding by either in-screen prediction or inter-screen prediction, and encodes the output order information of each image. A restoration step of decoding the compressed image data and restoring it to a reproduced image, an image storage step of storing one or more of the reproduced images as a reference image used for encoding a subsequent image, and the image. A control step for controlling a storage step is provided, and in the control step, the moving image prediction coding device is provided.
1) CRA picture at which decoding of coded data is started,
2) It is decoded after the CRA picture associated with the picture, output before the associated CRA picture, the decoding process by the restoration step is skipped, and it is not stored and output in the image storage step. A picture, at least one reference picture in which the decoding process by the restoration step is skipped in order to perform interscreen prediction, or at least one reference picture that precedes the associated CRA picture in the decoding order. The picture, which has a list of referenced pictures, including
3) A picture that is decoded by the restoration step and stored in the image storage step for reference as needed, and the picture has a list of reference pictures for interscreen prediction, and the reference. The picture, characterized in that all referenced pictures in the list of pictures are decoded by the restore step and are decoded after the CRA picture associated with the picture.
The image is classified and controlled in a total of three.

The moving image prediction coding program according to the present invention encodes a computer by an input means for inputting a plurality of images constituting a moving image and the image by either in-screen prediction or inter-screen prediction. A coding means for generating compressed image data including an image serving as a random access point and encoding the output order information of each image, and a restoration means for decoding the compressed image data and restoring it to a reproduced image. The reproduced image is made to function as an image storage means for storing one or more as a reference image used for encoding a subsequent image and a control means for controlling the image storage means.
1) CRA picture at which decoding of coded data is started,
2) A picture that is decoded after the CRA picture associated with the picture, output before the associated CRA picture, the decoding process by the restoration means is skipped, and the picture is not stored in the image storage means and is not output. The inter-screen prediction includes at least one reference picture in which the decoding process by the restoration means is skipped, or at least one reference picture that precedes the associated CRA picture in the decoding order. The picture, which has a list of referenced pictures,
3) A picture that is decoded by the restoration means and stored in the image storage means for reference as needed, and the picture has a list of reference pictures for performing interscreen prediction, and the reference picture. The picture, characterized in that all the reference pictures in the list are decoded by the restoring means and are decoded after the CRA picture associated with the picture.
The image is classified and controlled in a total of three.

The moving image prediction / decoding device according to the present invention is a compression including a random access image obtained by encoding a plurality of images constituting a moving image by either in-screen prediction or inter-screen prediction. An input means for inputting image data and coded data indicating the output order of each image, a restoration means for decoding the compressed image data and the coded data, and restoring the reproduced image and output order information. The control means includes an image storage means for storing one or more of the reproduced images as a reference image used for decoding a subsequent image, and a control means for controlling the restoration means.
1) CRA picture at which decoding of coded data is started,
2) A picture that is decoded after the CRA picture associated with the picture, output before the associated CRA picture, the decoding process by the restoration means is skipped, and the picture is not stored in the image storage means and is not output. The inter-screen prediction includes at least one reference picture in which the decoding process by the restoration means is skipped, or at least one reference picture that precedes the associated CRA picture in the decoding order. The picture, which has a list of referenced pictures,
3) A picture that is decoded by the restoration means and stored in the image storage means for reference as needed, and the picture has a list of reference pictures for performing interscreen prediction, and the reference picture. The picture, characterized in that all the reference pictures in the list are decoded by the restoring means and are decoded after the CRA picture associated with the picture.
The image is classified and controlled in a total of three, and when the decoding of the encoded data is started with the CRA picture associated with the picture, all of the reference pictures related to the picture are listed. It is determined whether or not the reference picture of is stored in the image storage means, and if all the reference pictures in the list of reference pictures are stored, the picture is decoded and one or more of the reference pictures in the list of reference pictures are stored. The decoding process, such as skipping the decoding of the picture if the reference picture of the above is not stored, is continued for a period until immediately before the processing of the next CRA picture.

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, and is based on either in-screen prediction or inter-screen prediction for a plurality of images constituting the moving image. An input step for inputting compressed image data including a random access image obtained by encoding and encoded data indicating the output order of each image, and the compressed image data and the encoded data. A restoration step of decoding and restoring the reproduced image and output order information, an image storage step of storing one or more of the reproduced images as a reference image used for decoding a subsequent image, and the restoration step. A control step for controlling is provided, and in the control step, the moving image prediction / decoding device is provided.
1) CRA picture at which decoding of coded data is started,
2) It is decoded after the CRA picture associated with the picture, output before the associated CRA picture, the decoding process by the restoration step is skipped, and it is not stored and output in the image storage step. A picture, at least one reference picture in which the decoding process by the restoration step is skipped in order to perform interscreen prediction, or at least one reference picture that precedes the associated CRA picture in the decoding order. The picture, which has a list of referenced pictures, including
3) A picture that is decoded by the restoration step and stored in the image storage step for reference as needed, and the picture has a list of reference pictures for interscreen prediction, and the reference. The picture, characterized in that all referenced pictures in the list of pictures are decoded by the restore step and are decoded after the CRA picture associated with the picture.
In a total of three, the image is classified and controlled, and in the restoration step, when the decoding of the encoded data is started with the CRA picture associated with the picture, the picture It is determined whether or not all the reference pictures in the list of reference pictures related to are stored in the image storage step, and if all the reference pictures in the list of reference pictures are stored, the picture is decoded. It is characterized in that the decoding process of skipping the decoding of the picture if one or more reference pictures in the list of the reference pictures are not stored is continued for a period until immediately before the processing of the next CRA picture.

The moving image prediction / decoding program according to the present invention is a random access image obtained by encoding a plurality of images constituting a moving image by either in-screen prediction or inter-screen prediction. An input means for inputting compressed image data including the image and encoded data indicating the output order of each image, decoding the compressed image data and the encoded data, and restoring the reproduced image and output order information. The means, the image storage means for storing one or more of the reproduced images as a reference image used for decoding the subsequent image, and the control means for controlling the restoration means are made to function, and the control means is used.
1) CRA picture at which decoding of coded data is started,
2) A picture that is decoded after the CRA picture associated with the picture, output before the associated CRA picture, the decoding process by the restoration means is skipped, and the picture is not stored in the image storage means and is not output. The inter-screen prediction includes at least one reference picture in which the decoding process by the restoration means is skipped, or at least one reference picture that precedes the associated CRA picture in the decoding order. The picture, which has a list of referenced pictures,
3) A picture that is decoded by the restoration means and stored in the image storage means for reference as needed, and the picture has a list of reference pictures for performing interscreen prediction, and the reference picture. The picture, characterized in that all the reference pictures in the list are decoded by the restoring means and are decoded after the CRA picture associated with the picture.
The image is classified and controlled in a total of three, and when the decoding of the encoded data is started with the CRA picture associated with the picture, all of the reference pictures related to the picture are listed. It is determined whether or not the reference picture of is stored in the image storage means, and if all the reference pictures in the list of reference pictures are stored, the picture is decoded and one or more of the reference pictures in the list of reference pictures are stored. The decoding process, such as skipping the decoding of the picture if the reference picture of the above is not stored, is continued for a period until immediately before the processing of the next CRA picture.

The moving image prediction / decoding device according to the present invention is compressed image data for a plurality of pictures constituting a moving image, and includes a random access picture and a random access skip (RAS) reading picture for each picture. An input means for inputting a bit stream containing compressed image data having information on the NAL unit type that identifies the picture as one of a plurality of picture types including the non-RAS reading picture, and the picture type. Based on the above, the compressed image data is decoded and restored to a reproduced image, and an output means for outputting the reproduced image is provided.
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
The reference picture set of the non-RAS reading picture, including the reference picture used for interscreen prediction of the non-RAS reading picture, precedes the RAS reading picture and the associated random access picture in decoding order. Does not include any of the pictures
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. It is characterized in that it does not include any of the preceding pictures.

Also, when the second random access picture is decoded after the first random access picture, it is used for interscreen prediction of other pictures following the second random access picture in the decoding order. The reference picture set of the second random access picture, including the pictures, may be characterized by not including any of the pictures preceding the first random access picture in the decoding order.

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, which is compressed image data for a plurality of pictures constituting a moving image, and is for each picture. , Has NAL unit type information that identifies the picture as one of a plurality of picture types including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture. An input step for inputting a bit stream including compressed image data, a restoration step for decoding the compressed image data based on the picture type and restoring the reproduced image, and an output step for outputting the reproduced image. Equipped with
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
The reference picture set of the non-RAS reading picture, including the reference picture used for interscreen prediction of the non-RAS reading picture, precedes the RAS reading picture and the associated random access picture in decoding order. Does not include any of the pictures
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. It is characterized in that it does not include any of the preceding pictures.

Also, when the second random access picture is decoded after the first random access picture, it is used for interscreen prediction of other pictures following the second random access picture in the decoding order. The reference picture set of the second random access picture, including the pictures, may be characterized by not including any of the pictures preceding the first random access picture in the decoding order.

The moving image prediction / decoding program according to the present invention is compressed image data for a plurality of pictures constituting a moving image on a computer, and for each picture, a random access picture and a random access skip (RAS). An input means for inputting a bit stream containing compressed image data having information on a NAL unit type that identifies the picture as one of a plurality of picture types including a reading picture and a non-RAS reading picture, and the above-mentioned. It functions as a restoration means for decoding the compressed image data and restoring it to a reproduced image based on the picture type, and an output means for outputting the reproduced image.
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
The reference picture set of the non-RAS reading picture, including the reference picture used for interscreen prediction of the non-RAS reading picture, precedes the RAS reading picture and the associated random access picture in decoding order. Does not include any of the pictures
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. It is characterized in that it does not include any of the preceding pictures.

Also, when the second random access picture is decoded after the first random access picture, it is used for interscreen prediction of other pictures following the second random access picture in the decoding order. The reference picture set of the second random access picture, including the pictures, may be characterized by not including any of the pictures preceding the first random access picture in the decoding order.

The moving image prediction / decoding device according to the present invention is compressed image data for a plurality of pictures constituting a moving image, and includes a random access picture and a random access skip (RAS) reading picture for each picture. An input means for inputting a bit stream containing compressed image data having information on the NAL unit type that identifies the picture as one of a plurality of picture types including the non-RAS reading picture, and the picture type. Based on the above, the compressed image data is decoded and restored to a reproduced image, and an output means for outputting the reproduced image is provided.
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. The RAS reading picture is not output if it does not include any of the preceding pictures and the associated random access picture is the first picture of the bitstream in the decoding order.

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, which is compressed image data for a plurality of pictures constituting a moving image, and is for each picture. , Has NAL unit type information that identifies the picture as one of a plurality of picture types including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture. An input step for inputting a bit stream including compressed image data, a restoration step for decoding the compressed image data based on the picture type and restoring the reproduced image, and an output step for outputting the reproduced image. Equipped with
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. The RAS reading picture is not output if it does not include any of the preceding pictures and the associated random access picture is the first picture of the bitstream in the decoding order.

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, which is compressed image data for a plurality of pictures constituting a moving image, and is for each picture. , Has NAL unit type information that identifies the picture as one of a plurality of picture types including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture. An input step for inputting a bit stream including compressed image data, a restoration step for decoding the compressed image data based on the picture type and restoring the reproduced image, and an output step for outputting the reproduced image. Equipped with
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
The reference picture set of the non-RAS reading picture, including the reference picture used for interscreen prediction of the non-RAS reading picture, precedes the RAS reading picture and the associated random access picture in decoding order. If none of the pictures are included and the second random access picture is decoded after the first random access picture, then the reference picture set of the second random access picture is said to be the first. The random access picture does not include any of the pictures preceding in the decoding order, and in the restoration step, the moving image prediction decoding device determines whether or not the picture is correctly decoded at the start of decoding the picture. , Characterized by.

In the determination of the restoration step, the moving image prediction decoding device determines whether or not the picture is the RAS reading picture, and if the picture is the RAS reading picture, it is determined that the picture is not correctly decoded. If the picture is not the RAS reading picture, it may be determined that the picture is correctly decoded.

In the determination of the restoration step, the moving image prediction / decoding device determines whether or not all the reference pictures in the list of reference pictures related to the picture are stored, and all the reference pictures in the list of reference pictures are stored. If it is stored, it is determined that the picture is correctly decoded, and if one or more reference pictures in the list of reference pictures are not stored, it is determined that the picture is not correctly decoded. ..

The moving image prediction / decoding method according to the present invention is a moving image prediction / decoding method executed by a moving image prediction / decoding device, which is compressed image data for a plurality of pictures constituting a moving image, and is for each picture. , Has NAL unit type information that identifies the picture as one of a plurality of picture types including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture. An input step for inputting a bit stream including compressed image data, a restoration step for decoding the compressed image data based on the picture type and restoring the reproduced image, and an output step for outputting the reproduced image. Equipped with
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. If none of the preceding pictures are included and the associated random access picture is the first picture in the bitstream in decoding order, no RAS reading picture is output and the decoding process is randomized in the restore step. When starting from an access picture, the non-RAS reading picture is decoded and the decoding of the RAS reading picture is skipped.

10 ... Recording medium, 30 ... Computer, 100 ... Moving image prediction coding device, 101 ... Input terminal, 102 ... Block divider, 103 ... Prediction signal generator, 104 ... Frame memory, 105 ... Subtractor, 106 ... Converter , 107 ... Quantifier, 108 ... Inverse Quantizer, 109 ... Inverse Converter, 110 ... Adder, 111 ... Entropy Encoder, 112 ... Output Terminal, 113 ... Input Terminal, 114 ... Frame Memory Manager, 200 ... Moving image prediction / decoding device, 201 ... Input terminal, 202 ... Data analyzer, 203 ... Inverse quantizer, 204 ... Inverse converter, 205 ... Adder, 206 ... Output terminal, 207 ... Frame memory, 208 ... Prediction signal Generator, 209 ... Frame memory manager, 210 ... Control, P100 ... Video prediction coding program, P101 ... Input module, P102 ... Coding module, P103 ... Restoration module, P104 ... Image storage module, P105 ... Control module , P200 ... Video prediction / decoding program, P201 ... Input module, P202 ... Restoration module, P203 ... Image storage module, P204 ... Control module.

Claims (1)

A moving image predictive decoding method executed by a moving image predictive decoding device.
A plurality of compressed image data for a plurality of pictures constituting a moving image, including a random access picture, a random access skip (RAS) reading picture, and a non-RAS reading picture for each picture. An input step that inputs a bit stream containing compressed image data with information on the NAL unit type that identifies the picture as one of the picture types.
A restoration step of decoding the compressed image data and restoring it to a reproduced image based on the picture type.
The output step to output the reproduced image and
Equipped with
1) The random access picture is the first picture of the bitstream in the decoding order when the decoding process started from any random access picture of the bitstream is started from the random access picture.
2) The RAS reading picture is a picture that precedes the related random access picture in the output order and cannot be decoded if the related random access picture is the first picture of the bitstream in the decoding order. Yes,
3) The non-RAS reading picture is a picture that precedes the related random access picture in the output order and can be decoded.
The reference picture set of the non-RAS reading picture, including the reference picture used for interscreen prediction of the non-RAS reading picture, precedes the RAS reading picture and the associated random access picture in decoding order. Does not include any of the pictures
If the second random access picture is decoded after the first random access picture, the reference picture set of the second random access picture will be decoded in order to the first random access picture. Does not include any preceding pictures
In the restore step, if the decoding process is started with a random access picture, the non-RAS reading picture is decoded and the decoding of the RAS reading picture is skipped.
A moving image prediction decoding method characterized by this.

Images