JP2004228617A - Dynamic image encoder and dynamic image decoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a dynamic image encoder and a dynamic image decoder wherein a capacity required for a memory can be reduced. <P>SOLUTION: The dynamic image encoder includes a reference enabled picture discrimination section 112 enabling one encoded picture or more stored in a storage area to be referenced, and the reference enabled picture discrimination section 112 estimates an operation of a storage area in a corresponding decoder and discriminates the possibility of referencing so as to reserve a storage area required for decoding processing for a picture being a decoding object by the decoder into the storage area of the decoder. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a video encoding device and a video decoding device, and more particularly to a video encoding device and a video decoding device that perform inter-screen prediction with reference to already coded pictures. .
[0002]
[Prior art]
In recent years, the multimedia era, in which voices, images, and other pixel values are integrated, has been approached, and the traditional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, and telephones to humans, is a multimedia technology. It has been taken up as an object. Generally, multimedia means not only characters, but also figures, sounds, and especially images, etc., that are simultaneously associated with each other. Is an essential condition.
[0003]
However, when the amount of information of each of the above information media is estimated as a digital information amount, the amount of information per character is 1 to 2 bytes in the case of characters, whereas 64 kbits per second in the case of voice (telephone quality) In addition, for a moving image, an information amount of 100 Mbits per second (current television reception quality) or more is required, and it is not realistic to handle the vast amount of information in the above-mentioned information medium in a digital form. For example, a videophone has already been put into practical use by an Integrated Services Digital Network (ISDN) having a transmission rate of 64 kbps to 1.5 Mbps, but it is impossible to send a video of a video camera directly through the ISDN. It is possible.
[0004]
Therefore, what is needed is an information compression technology. For example, in the case of a videophone, H.264 standardized internationally by ITU (International Telecommunication Union Telecommunication Standardization Sector). 261 and H.E. H.263 video compression technology is used. In addition, according to the information compression technology of the MPEG-1 standard, it is possible to store image information together with audio information in a normal music CD (compact disc).
[0005]
Here, MPEG (Moving Picture Experts Group) is an international standard for digital compression of moving picture plane signals, and MPEG-1 converts moving picture plane signals up to 1.5 Mbps, that is, information of television signals by about 1/100. It is a standard that compresses up to. In addition, since the transmission rate for the MPEG-1 standard is mainly limited to about 1.5 Mbps, the moving picture signal is 2 to 2 in MPEG-2, which is standardized to meet the demand for higher image quality. It is compressed to 15 Mbps.
[0006]
Further, at present, MPEG-4, which has a higher compression ratio, has been standardized by a working group (ISO / IEC JTC1 / SC29 / WG11), which has been standardizing MPEG-1 and MPEG-2. MPEG-4 initially introduced not only high-efficiency coding at low bit rates, but also a strong error resilience technique that can reduce subjective image quality degradation even if a transmission path error occurs. . Currently, H.264 is the next-generation screen coding scheme jointly developed by ISO / IEC and ITU. H.264 standardization activities are underway.
[0007]
Generally, in coding of a moving image, the amount of information is compressed by reducing redundancy in the time direction and the space direction. Therefore, in the inter-picture prediction coding for the purpose of reducing temporal redundancy, motion detection and prediction image creation are performed in block units with reference to the forward or backward picture, and the obtained predicted image and the current Encoding is performed on the difference value from the picture.
[0008]
Here, a picture is a term representing one screen, and means a frame in a progressive image and a frame or a field in an interlaced image. Here, an interlaced image is an image in which one frame is composed of two fields at different times. In encoding and decoding of an interlaced image, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can.
[0009]
Note that the picture described below is described in terms of a frame in a progressive image, but the description can be similarly applied to a frame or a field in an interlaced image.
[0010]
FIG. 40 is a diagram for explaining types of pictures and their reference relationships.
A picture that does not have a reference picture and performs intra-picture predictive encoding, such as picture I0, is called an I picture. A picture such as the picture P6, which refers to only one picture at a time and performs inter-picture predictive coding, is called a P picture. A picture that can perform inter-picture predictive coding by simultaneously referring to two pictures is called a B picture. A B picture can refer to two pictures in arbitrary directions in time, such as pictures B2 and B5. However, as a condition for encoding and decoding these pictures, the picture to be referred to must be already encoded and decoded. Also, since B5 is encoded with reference to an encoded B picture, even in the case of such a reference relationship, it is necessary to store even a B picture in a reference memory. FIG. 40 (a) shows the display order, and FIG. 40 (b) rearranges (a) in the order of encoding and decoding. It can be seen that all pictures referenced by pictures B2 and P6 have been rearranged as if they had been encoded and decoded first.
[0011]
The encoded and decoded pictures are stored in a picture memory. For example, in the case of encoding P6 in FIG. 40, I0, P3, B1, and B2 are stored in the picture memory, and one optimal picture is selected from each of the blocks and predicted as a reference picture. Encoding can be performed. Although P3 is selected in the example of FIG. 40, it is possible to refer to a different picture for each block within the same picture.
[0012]
Each picture stored in the picture memory is provided with information indicating whether the picture can be used for reference or not. Originally, pictures not used for reference should be deleted from the picture memory immediately. For example, as shown in P3 of FIG. 40, the decoding order is the second, but the display order is the fourth. Since it is necessary to wait for another picture to be decoded until the display order comes, it is necessary to store the picture in the picture memory even if it is known that the picture is not used for reference.
[0013]
Hereinafter, the conventional encoding and decoding methods will be described in the order of (1) the operation of the encoding device, (2) the operation of the decoding device, and (3) the picture memory management method.
[0014]
(1) Operation of coding apparatus
FIG. 33 is a block diagram of a conventional image encoding device. The image encoding apparatus includes a picture memory 101, a prediction residual encoding unit 102, a code sequence generation unit 103, a prediction residual decoding unit 104, a picture memory 105, a motion vector detection unit 106, a motion compensation encoding unit 107, It comprises a motion vector storage unit 108, a picture memory control unit 109, a working picture memory 111, and the like.
[0015]
The moving image to be encoded is input to the picture memory 101 in units of pictures in the order of display, and the pictures are rearranged in the order of encoding as shown in FIG. Further, each picture is divided into blocks of, for example, 16 × 16 pixels, which are called macroblocks, and the subsequent processing is performed in block units.
[0016]
An input image signal read from the picture memory 101 is input to a difference calculation unit 113, and a difference image signal obtained by taking a difference from a prediction image signal output from the motion compensation coding unit 107 is converted to a prediction residual code. Output to the conversion unit 102. The prediction residual encoding unit 102 performs image encoding processing such as frequency conversion and quantization, and outputs a residual signal. The residual signal is input to the prediction residual decoding unit 104, and performs image decoding processing such as inverse quantization and inverse frequency conversion, and outputs a residual decoded signal. The addition operation unit 114 generates a reconstructed image signal by adding the decoded residual signal and the predicted image signal, and stores the obtained reconstructed image signal in the working picture memory 111. When the encoding of one picture is completed, the encoded picture in the working picture memory 111 is stored in the picture memory 105.
[0017]
On the other hand, the input image signal in macroblock units read from the picture memory 101 is also input to the motion vector detection unit 106. Here, one or a plurality of encoded pictures stored in the picture memory 105 are searched, and a motion vector indicating the position by detecting an image area closest to the input image signal is selected. And determine a reference picture index pointing to the obtained picture. The motion vector detection is performed for each block obtained by further dividing the macroblock, and the obtained motion vector is stored in the motion vector storage unit 108. The motion compensation coding unit 107 extracts an optimal image area from the coded pictures stored in the picture memory 105 and generates a predicted image using the motion vector detected by the above processing.
[0018]
By performing variable length coding in the code sequence generation unit 103 on the coded information such as the motion vector, the reference picture index, and the residual coded signal output by the above series of processes, the coding apparatus outputs Is obtained.
[0019]
The flow of the processing described above is the operation in the case where the inter-picture prediction coding is performed. When performing intra-frame encoding, a predicted image is not generated by motion compensation, and a differential image signal is generated by generating a predicted image of an encoding target region from an encoded region in the same screen and taking a difference. . The difference image signal is converted into a residual coded signal in a prediction residual coder 102 and subjected to variable-length coding in a code sequence generator 103, as in the case of inter-picture prediction coding, and is output. Is obtained.
[0020]
(2) Operation of decoding device
FIG. 34 is a block diagram of a conventional image decoding device. The image decoding apparatus includes a code sequence analysis unit 201, a prediction residual decoding unit 202, a picture memory 203, a motion compensation decoding unit 204, a motion vector storage unit 205, a picture memory control unit 206, a working picture memory 207, and the like. Consists of
[0021]
First, various information such as a motion vector and a residual coded signal are extracted from the input code sequence by the code sequence analyzing unit 201. The motion vector extracted by the code sequence analyzer 201 is output to the motion vector storage unit 205, and the residual coded signal is output to the prediction residual decoding unit 202.
[0022]
The prediction residual decoding unit 202 performs image decoding processing such as inverse quantization and inverse frequency transform on the input residual encoded signal, and outputs a residual decoded signal. The addition operation unit 209 generates a reconstructed image signal by adding the residual decoded signal and the predicted image signal output from the motion compensation encoding unit 204, and stores the obtained reconstructed image signal in a working picture memory. 207. When decoding of one picture is completed, the decoded picture in the working picture memory 207 is stored in the picture memory 203.
[0023]
The motion compensation decoding unit 204 uses the motion vector input from the motion vector storage unit 205 and the reference picture index input from the code sequence analysis unit 201 to store one or more frames stored in the picture memory 203. An optimal image area for the predicted image is extracted from the decoded picture.
[0024]
The decoded picture generated by the above series of processing is output as a display image signal in accordance with the timing displayed from the picture memory 203.
The above processing flow is an operation in the case where the inter-picture prediction decoding is performed. However, the switch 210 switches between the intra-picture prediction decoding. When performing intra-screen decoding, a predicted image is not generated by motion compensation, and a reconstructed image signal is generated by generating a predicted image of a decoding target region from a decoded region in the same screen and performing addition. Then, the obtained reconstructed image signal is stored in the working picture memory 207. When decoding of one picture is completed, the decoded picture in the working picture memory 207 is stored in the picture memory 203 and output as a display image signal in accordance with the display timing.
[0025]
(3) Picture memory management method
H. is currently standardizing. Since H.264 allows a very complicated reference relationship as described above, a method for managing a plurality of decoded pictures in the picture memory 203 is defined.
[0026]
Note that each decoded picture is stored in the picture memory according to the encoding order, but has information indicating the display order separately from the picture memory.
The outline of the picture memory management is as follows.
[0027]
First, when decoding of the target picture is completed, designation is made as to whether the decoded picture already stored in the picture memory can be referred to or not. H. There are two methods for the designation. H.264. The first method is a method called a sliding window, in which, when a certain number of referenceable pictures in a picture memory have reached a certain number, reference to the oldest coding order is designated as non-referenceable. The second method is a method called MMCO, which makes it possible to designate a reference to an arbitrary picture by using a command. Next, a process of deleting the decoded picture in the picture memory is performed, and the decoded picture stored in the working memory is stored in an empty area. Finally, designation of referencing or non-referencing of the stored picture is performed according to the information described in the header, and the process proceeds to the decoding processing of the next picture.
[0028]
FIG. 37 shows the flow of processing of the decoding apparatus when managing the picture memory using the above-mentioned Siliding Window, and shows the operation of the picture memory control unit 206 in the block diagram of FIG.
[0029]
The target picture is decoded while being stored in the working storage area. After the decoding is completed, if the number of referenceable pictures in the picture memory has reached a certain number, the referenceable picture with the oldest encoding order cannot be referred to. Specify. Next, it is determined whether or not there is a free area in the picture memory. If there is no free area, it is further determined whether or not there is a picture in the picture memory (storage area) which is designated as non-referenceable. I do. If there is a non-referenceable picture, the oldest non-referenceable picture with display order information is deleted, and if there is no non-referenceable picture, the oldest referenceable picture with display order information is deleted. Then, the decoded picture is stored in the vacant area secured by the above processing, and the picture is designated (marked) as referable or non-referable based on the header information.
[0030]
FIG. 35 shows the flow of processing of the encoding device when managing the picture memory using the above-mentioned Siliding Window, and shows the operation of the picture memory control unit 109 in the block diagram of FIG.
[0031]
The target picture is coded by referring to all the referenceable pictures, and if the referenceable picture in the picture memory has reached a certain number after completion of the reference, the referenceable picture having the oldest coding order cannot be referred to. And Then, the coded picture is stored in the picture memory, and designation (marking) of referring to or not referencing the picture is performed. The method of deleting the coded picture in the picture memory in the coding device is arbitrary except that only a non-referenceable picture can be deleted, and can be freely controlled according to the system configuration.
[0032]
FIG. 38 shows the flow of processing of the decoding device when the picture memory is managed using the MMCO, and shows the operation of the picture memory control unit 206 in the block diagram of FIG. The difference from FIG. 37 is that by analyzing the management command input from the code string analysis unit 201 before designating a non-referenceable picture, it becomes possible to designate non-referenceable for any picture. The other operations are exactly the same as those described above.
[0033]
FIG. 36 shows a flow of processing of the encoding device when the picture memory is managed using the MMCO, and shows the operation of the picture memory control unit 109 in the block diagram of FIG. The only difference from FIG. 35 is that the memory management command generation unit 110 generates a management command. The other operations are exactly the same as those described above. In FIG. 36, the management command is generated before the encoding. However, the management command can be handled in the same manner even if the order is reversed.
[0034]
Whether to use the sliding window or the MMCO is specified by a flag described in a slice header. FIG. 39 shows an example of the code string at that time. Whether the sliding window or the MMCO is used is specified by a flag mmco_flg described in the slice header area. When the MMCO is used, a management command mmco_cmd is described after mmco_flg.
[0035]
[Problems to be solved by the invention]
Although the picture memory management method described in the related art can manage based on a very simple rule, the timing of deleting the decoded picture in the picture memory in the decoding device is determined by the decoding of the target picture. Since decoding is completed, an area for storing intermediate decoded data cannot be prepared in the picture memory when the current picture is being decoded. Therefore, as the working picture memory 111 in FIG. 33 and the working picture memory 207 in FIG. 34, a memory having a capacity to store one picture needs to be prepared separately from the picture memory. The total amount of memory in the device will increase significantly.
[0036]
Therefore, the present invention restricts the pictures that can be referred to at the time of encoding in an encoding device, so that a decoding device decodes a target picture while storing intermediate decoded data in a picture memory. It is an object of the present invention to reduce the required capacity of the memory in the moving picture encoding device and the moving picture decoding device by making the work picture memory unnecessary.
[0037]
[Means for Solving the Problems]
In order to achieve this object, a moving picture coding apparatus according to the present invention is a moving picture decoding apparatus for coding an input image with reference to an already coded picture, Reference-picture determining means for determining a picture which can be referred to one or a plurality of coded pictures stored in the decoding apparatus, and the reference-capable picture determining means includes The operation of the storage area is assumed, and it is determined that the picture to be decoded in the decoding device can be referred to so that the storage area necessary for the decoding process can be secured in the storage area. It is characterized by being.
[0038]
Further, in another moving picture coding apparatus according to the present invention, the referenceable picture determination means prohibits reference to a picture deleted after decoding the current picture from a storage area of the assumed decoding apparatus. It is characterized by the following.
[0039]
Further, in another moving picture encoding apparatus according to the present invention, the referenceable picture determination means refers to a picture having the oldest display order information among pictures stored in a storage area of the assumed decoding apparatus. Is prohibited.
[0040]
Further, another moving image encoding apparatus according to the present invention is an apparatus for encoding an input image with reference to an already encoded picture, wherein one or a plurality of images stored in a storage area are stored. Has a memory management command generating means for generating a command prohibiting the use of a coded picture for reference, and wherein the memory management command generating means assumes an operation of a storage area in a corresponding decoding device. A decoding device that generates a command such that a picture deleted from the storage area after decoding processing of a picture to be decoded by the decoding apparatus is always inaccessible.
[0041]
Further, the moving picture decoding apparatus according to the present invention is a moving picture decoding apparatus that decodes a code sequence generated by performing encoding with reference to an encoded picture, and is stored in a storage area. A writable area determining unit that determines an area that can be overwritten on the decoded picture during the decoding process of the decoding target picture; and an area that is determined to be writable by the writable area determining unit. The decoding of the target picture is performed while storing the decoded data.
[0042]
Further, in another moving picture decoding apparatus according to the present invention, the writable area determination unit may determine, as a writable area, an area in which a picture to be deleted from a storage area after decoding processing of the current picture is stored. It is characterized by.
[0043]
Further, in another moving picture decoding apparatus according to the present invention, the writable area determination unit may display the writable area in a state where it is possible to determine whether the decoded picture in the storage area has already been displayed. It is characterized in that, among the pictures determined to be, the area in which the pictures which cannot be referred to is stored is determined as a writable area.
[0044]
Further, another moving picture decoding apparatus according to the present invention is characterized in that the writable area determination means determines an area in which a picture having the oldest display order information in the storage area is stored as a writable area. And
[0045]
BEST MODE FOR CARRYING OUT THE INVENTION
(Embodiment 1)
Embodiment 1 of the present invention describes a moving picture coding method and a decoding method when picture memory management is performed by a sliding window, and is a block diagram of the coding apparatus shown in FIG. 1 and FIG. And the flowchart of the encoding device shown in FIGS. 3 and 4, the flowchart of the decoding device shown in FIGS. 8 and 9, and the pictures shown in FIGS. 13 to 17. First, an outline of encoding and decoding will be described with reference to a schematic diagram of a memory, and then details of encoding and decoding based on picture memory management will be described for each path in the flowchart.
[0046]
FIG. 2 is a block diagram of the decoding device according to the first embodiment. The configuration of the decoding device and the flow of processing are almost the same as those of the conventional method described with reference to FIG. The difference from the conventional method is that the writable area determination unit 208 is added and the working picture memory 207 is removed.
[0047]
In the conventional method, after the decoding of the target picture is completed, the picture memory control unit 206 designates the picture in the picture memory 203 as referable and non-referable, and further performs the picture deletion processing. After that, a procedure of storing the target decoded picture in the picture memory 203 was performed. Therefore, during decoding, data cannot be stored in the picture memory 203, so that a working picture memory 207 is prepared separately from the picture memory 203 to perform decoding.
[0048]
On the other hand, in the present embodiment, it is necessary to analyze beforehand how the picture memory is managed before decoding the current picture and to overwrite the pictures stored in the picture memory 203. Is determined by the writable area determination unit 208, and the decoded data generated while decoding is sequentially stored in the picture memory 203 while being overwritten. Therefore, decoding can be performed without the need for the working picture memory 207, so that the memory in the decoding device can be significantly reduced.
[0049]
In particular, the working picture memory is H.264. Since it is not clearly defined by the H.264 standard, it is necessary to set the memory capacity assuming the largest possible picture size to be input, which is a great obstacle to implementation. On the other hand, since the capacity of the picture memory is clearly defined in the standard, the present embodiment that enables all decoding processes to be performed within the capacity simplifies the implementation of the decoding device. It is possible.
[0050]
FIG. 1 is a block diagram of an encoding device according to Embodiment 1. The configuration of the encoding device and the flow of processing are almost the same as those of the conventional method described with reference to FIG. The difference from the conventional method is that the referenceable picture determination unit 112 is added and the working picture memory 111 is removed.
[0051]
In the decoding device according to the present embodiment, since the decoded picture data is stored as it is in the picture memory while being overwritten, it is possible to overwrite when encoding the picture with the coding device. An encoding method is required on the premise that an area is secured in the picture memory. That is, instead of performing coding by simply referring to the picture stored in the picture memory 105 based on the information indicating whether or not the picture can be referred to as in the conventional method, the picture memory management of the decoding apparatus is performed. Assuming a method at the time of encoding, it is necessary to specify which picture is overwritten at the time of decoding, and to perform encoding without referring to the picture at the time of encoding. The referable picture determination unit 112 specifies a picture that is prohibited from being referred to, and when the motion vector detection unit 106 refers to an encoded picture in the picture memory 105, whether the reference is possible. To the motion vector detection unit 106.
[0052]
In the present embodiment, since the picture memory is managed by the sliding window, the management command is not actually generated by the memory management command generation unit 110, and the flag for indicating the use of the sliding window is sliced. Only described in the header.
[0053]
FIG. 27 shows an example of a code string generated by the coding device according to the present embodiment. As in the conventional method, the flag mmco_flg specifies that the picture memory is managed using the sliding window. The management command mmco_cmd is data necessary only when MMCO is used, and in the present embodiment, the management command mmco_cmd is not required. On the other hand, as a point different from the conventional method, a flag dpb_flg is described in the picture information area. This flag is a flag indicating that the code string can be decoded only by the picture memory by the method described in the present embodiment when decoding. The flag dpb_flg is not necessarily required for decoding in a decoding device having a conventional configuration, but the decoding device having the configuration described in the present embodiment analyzes the flag. This makes it possible to determine whether decoding is possible only with the picture memory. The flag dpb_flg is described in the picture common information area in the example of FIG. 27, but may be described in the sequence common information area or another information area.
[0054]
Next, details of the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device, and the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device will be described with reference to FIGS. An explanation will be given for each of these routes using the flowchart shown in FIG.
[0055]
(Route 1)
FIG. 8 is a flowchart for explaining operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 1 will be described. FIG. 13 is a diagram for explaining how the state of the picture memory in path 1 changes. In the example shown in the figure, six pictures can be stored, and the pictures are stored from left to right in order of decoding. P0, P3, B1, and B2 represent the pictures shown in FIG. 40, and the numbers indicate the display order. used indicates a picture that can be referred to, and unused indicates a picture that cannot be referred to. Also, Current and C indicate data of a decoding target picture. Note that the vertical dotted line in the drawing indicates the boundary between a referenceable picture and a non-referenceable picture, and in the example of the drawing, it is possible to hold up to five referenceable pictures.
[0056]
As shown in FIG. 8, first, before starting decoding the current picture, the writable area determination unit 208 determines whether there is a free area in the picture memory 203 where no decoded picture is stored. FIG. 13A shows the state of the picture memory before decoding is started. According to this figure, it can be seen that there are two writing areas in the picture memory. According to the path 1, if there is a free area, it is instructed to decode the target picture while storing the data being decoded in that area. In other words, the decoding of the current picture is performed while the data being decoded is stored in the free area designated by the writable area determination unit 208 as shown in FIG. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management is performed by a sliding window. Originally, designation of non-reference and deletion of pictures are performed in FIG. 13C. However, in the example of FIG. 13, these operations are not performed because the number of referenceable pictures has not reached five. FIG. 13D shows the state of the picture memory after the memory management processing is applied. In the drawing, since pictures in the decoding order from the left are accumulated from the left, the data is described for convenience as if the data were rearranged from FIG. 13 (c) to FIG. 13 (d). It does not move data. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all of these processes are completed, a memory state as shown in FIG. 13E is obtained, and is used for reference in decoding the next picture.
[0057]
On the other hand, FIG. 3 is a flowchart for explaining the operations of the picture memory control unit 109 and the referable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 1 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 13 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0058]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 13A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there are two writing areas in the assumed picture memory. According to the path 1, when there is a free area, it is instructed to encode the target picture while referring to all the referenceable pictures. That is, it is possible to perform encoding using all four referenceable pictures P0, P3, B1, and B2. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management is performed by a sliding window. Originally, designation of non-reference is performed in FIG. 13C, but in the example of FIG. 13, these operations are not performed because the number of referenceable pictures has not reached five. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the current picture can be referenced.
[0059]
(Route 2)
FIG. 8 is a flowchart for explaining operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 2 will be described. FIG. 14 is a diagram for explaining how the state of the picture memory changes in the path 2.
[0060]
First, before starting decoding of the current picture, the writable area determination unit 208 determines whether or not there is an empty area in the picture memory 203 where no decoded picture is stored. FIG. 14A shows the state of the picture memory before decoding is started. According to this figure, it can be seen that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory (storage area). FIG. 14A shows that P0 is a non-referenceable picture. Next, it is determined whether there is a non-referenceable picture having display order information whose decoding order information is older than the oldest referenceable picture. According to FIG. 14A, the display order information of the referenceable picture P3 having the oldest decoding order information is 3, whereas the non-referenceable picture is only P0 and the display order information thereof is 0. Has the oldest display order information. As a result, according to the path 2, it is instructed to decode the target picture while storing the data being decoded in the area of the unreferenceable picture having the oldest display order information. That is, the decoding of the target picture is performed while the data being decoded is stored in the non-referenceable picture area P0 having the oldest display order information designated by the writable area determination unit 208 as shown in FIG. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the picture memory has reached 5, it is specified (marking) that the referenceable picture P3 having the oldest decoding order information cannot be referenced as shown in FIG. Is done. Thereafter, as described in the related art with reference to FIG. 37, the reference-disabled picture having the oldest display order information is deleted. That is, in the example of FIG. 14C, P0 and P3 are deletion candidates, but P0 is deleted because the display order information is older for P0. However, physically, the picture to be decoded has already been overwritten on P0, so there is no need to delete data only by deleting management information. FIG. 14D shows the state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the target picture, it is determined whether the target picture can be referred to or cannot be referred to. As shown in FIG. Do. When all these processes are completed, a memory state as shown in FIG. 14E is obtained, and is used for reference in decoding the next picture.
[0061]
On the other hand, FIG. 3 is a flowchart for explaining the operations of the picture memory control unit 109 and the referable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 2 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 14 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0062]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 14A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. FIG. 14A shows that P0 is a non-referenceable picture. Next, it is determined whether there is a non-referenceable picture having display order information whose code order information is older than the oldest referenceable picture. According to FIG. 14A, the display order information of the referenceable picture P3 having the oldest code order information is 3, whereas the non-referenceable picture is only P0 and the display order information is 0. Has the oldest display order information. As a result, according to the path 2, it is instructed to encode the target picture while referring to all the referenceable pictures. That is, it is possible to perform encoding using all five referenceable pictures P3, B1, B2, P6, and B4. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the assumed picture memory has reached 5, it is specified that the referenceable picture P3 having the oldest decoding order information is made unreferenceable as shown in FIG. . In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0063]
(Route 3)
FIG. 8 is a flowchart for explaining operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 3 will be described. FIG. 15 is a diagram for explaining how the state of the picture memory in path 3 changes.
[0064]
First, before starting decoding of the current picture, the writable area determination unit 208 determines whether or not there is an empty area in the picture memory 203 where no decoded picture is stored. FIG. 15A shows the state of the picture memory before decoding is started, and it can be seen from this figure that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. FIG. 15A shows that P3 is a non-referenceable picture. Next, it is determined whether there is a non-referenceable picture having display order information whose decoding order information is older than the oldest referenceable picture. According to FIG. 15A, the display order information of the referenceable picture B1 having the oldest decoding order information is 1, whereas the non-referenceable picture is only P3 and the display order information is 3, so that the referenceable picture is Has the oldest display order information. As a result, according to the path 3, it is instructed to decode the target picture while storing the data being decoded in the area of the referenceable picture having the oldest decoding order information. That is, as shown in FIG. 15B, the current picture is decoded while the data being decoded is stored in the area B1 of the referenceable picture having the oldest decoding order information designated by the writable area determination unit 208. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the picture memory has reached 5, it is specified that the referenceable picture B1 having the oldest decoding order information is made unreferenceable as shown in FIG. Thereafter, as described in the related art with reference to FIG. 37, the reference-disabled picture having the oldest display order information is deleted. That is, in the example of FIG. 15C, P3 and B1 are deletion candidates, but B1 is deleted because the display order information of B1 is older. However, physically, since the decoding target picture has already been overwritten on B1, it is not necessary to delete data only by deleting management information. FIG. 15D shows a state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all these processes are completed, a memory state as shown in FIG. 15E is obtained, and is used for reference in decoding the next picture.
[0065]
On the other hand, FIG. 3 is a flowchart for explaining the operations of the picture memory control unit 109 and the referable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 3 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 15 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0066]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 15A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. FIG. 15A shows that P3 is a non-referenceable picture. Next, it is determined whether there is a non-referenceable picture having display order information whose code order information is older than the oldest referenceable picture. According to FIG. 15A, the display order information of the referenceable picture B1 having the oldest code order information is 1, whereas the non-referenceable picture is only P3 and the display order information is 3, so that the referenceable picture is Has the oldest display order information. As a result, according to the path 3, it is instructed to perform the encoding without referring to the referenceable picture whose encoding order information is the oldest. That is, encoding is performed without referring to B1 among the five referenceable pictures B1, B2, P6, B4, and B5. This is because B1 is overwritten during decoding during decoding and cannot be used for reference. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the assumed picture memory has reached 5, it is specified that the referenceable picture B1 having the oldest decoding order information is made unreferenceable as shown in FIG. . In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0067]
(Route 4)
FIG. 8 is a flowchart for explaining operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 4 will be described. FIG. 16 is a diagram for explaining how the state of the picture memory in path 4 changes. In FIG. 16, the maximum number of referenceable pictures that can be held is six.
First, before starting decoding of the current picture, the writable area determination unit 208 determines whether or not there is an empty area in the picture memory 203 where no decoded picture is stored. FIG. 16A shows the state of the picture memory before the start of decoding, and it can be seen from this figure that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. According to FIG. 16A, it can be seen that there is no non-referenceable picture. As a result, according to the path 4, it is instructed to decode the target picture while saving the data being decoded in the area of the referenceable picture having the oldest decoding order information. In other words, the current picture is decoded while the data being decoded is stored in the area P3 of the referenceable picture having the oldest decoding order information designated by the writable area determination unit 208 as shown in FIG. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the picture memory has reached 6, it is specified that the referenceable picture P3 having the oldest decoding order information is made unreferenceable as shown in FIG. Thereafter, as described in the related art with reference to FIG. 37, the reference-disabled picture having the oldest display order information is deleted. That is, in the example of FIG. 16C, only P3 is designated as non-referenceable, so P3 is deleted. However, physically, the picture to be decoded has already been overwritten on P3, so there is no need to delete data only by deleting management information. FIG. 16D shows the state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Do. When all these processes are completed, a memory state as shown in FIG. 16E is obtained, and is used for reference in decoding the next picture.
[0068]
On the other hand, FIG. 3 is a flowchart for explaining the operations of the picture memory control unit 109 and the referable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 4 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 16 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes. In FIG. 16, the maximum number of referenceable pictures that can be held is six.
[0069]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 16A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. According to FIG. 16A, it can be seen that there is no non-referenceable picture. As a result, according to the path 4, it is instructed to perform the encoding without referring to the referenceable picture whose encoding order information is the oldest. That is, encoding is performed without referring to P3 among the six referable pictures P3, B1, B2, P6, B4, and B5. This is because P3 is overwritten during decoding during decoding and cannot be used for reference. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the assumed picture memory has reached 6, it is specified that the referenceable picture P3 having the oldest decoding order information is made unreferenceable as shown in FIG. . In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0070]
(Route 2a)
FIG. 9 is a flowchart for explaining operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. The only difference from the operation of the flowchart in FIG. 8 is that the picture memory control unit 206 has a function of determining whether a decoded picture in the picture memory has already been displayed. Therefore, the route 1, the route 3, and the route 4 are completely the same as the embodiment described with reference to FIG. Here, the path 2a will be described. FIG. 17 is a diagram for explaining how the state of the picture memory changes in the path 2a.
[0071]
First, before starting decoding of the current picture, the writable area determination unit 208 determines whether or not there is an empty area in the picture memory 203 where no decoded picture is stored. FIG. 17A shows the state of the picture memory before the decoding is started. According to this figure, it can be seen that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. FIG. 17A shows that P3 is a non-referenceable picture. Next, it is determined whether or not the decoding order information has display order information that is older than the oldest referenceable picture or there is an undisplayable picture that has already been displayed. According to FIG. 17 (a), the display order information of the referenceable picture B1 having the oldest decoding order information is 1, whereas the non-referenceable picture is only P3 and the display order information is 3. It can be seen that the older one has the older display order information. However, P3, B1, and B2 are displayed pictures, and P3 is a non-referenceable displayed picture. As a result, according to the path 2a, it is instructed to decode the target picture while storing the data being decoded in the area of the non-referenceable picture having the oldest display order information. That is, as shown in FIG. 17B, the current picture is decoded while the data being decoded is stored in the area P3 of the non-referenceable picture having the oldest display order information designated by the writable area determination unit 208. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the picture memory has reached 5, it is specified that the referenceable picture B1 having the oldest decoding order information is made unreferenceable as shown in FIG. Thereafter, as described in the related art with reference to FIG. 37, the reference-disabled picture having the oldest display order information is deleted. That is, in the example of FIG. 17C, P3 and B1 are deletion candidates, but B1 is deleted because the display order information is older for B1. Also, since P3 has already been overwritten by the picture to be decoded, the originally stored data as P3 has already been lost. Therefore, P3 is removed from the picture memory state after application of the memory management processing shown in FIG. 17D. Finally, based on the information described in the header area of the target picture, it is determined whether the target picture can be referred to or cannot be referred to. As shown in FIG. Do. When all of these processes are completed, a memory state as shown in FIG. 17E is obtained, and is used for reference in decoding the next picture.
[0072]
On the other hand, FIG. 4 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. The only difference from the operation of the flowchart in FIG. 3 is that the picture memory control unit 109 has a function of determining whether a decoded picture in the assumed picture memory has already been displayed. Therefore, the route 1, the route 3, and the route 4 are completely the same as the embodiment described with reference to FIG. Here, the path 2a will be described. FIG. 17 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0073]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 17A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. FIG. 17A shows that P3 is a non-referenceable picture. Next, it is determined whether or not the code order information has display order information that is older than the oldest referenceable picture or there is an already-displayed non-referenceable picture. According to FIG. 17A, the display order information of the referenceable picture B1 whose code order information is the oldest is 1, whereas the non-referenceable picture is only P3 and the display order information is 3, so that the referenceable picture is Has the oldest display order information. However, P3, B1, and B2 are displayed pictures, and P3 is a non-referenceable displayed picture. As a result, according to the path 2a, it is instructed to encode the target picture while referring to all the referenceable pictures. That is, it is possible to perform encoding using all five referenceable pictures B1, B2, P6, B4, and B5. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management is performed by a sliding window. Since the number of referenceable pictures in the assumed picture memory has reached 5, it is specified that the referenceable picture B1 having the oldest decoding order information is made unreferenceable as shown in FIG. . In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0074]
In the embodiment of the present invention, in any of the descriptions of the paths, the work of designating the reference and non-reference of the picture to be encoded and decoded is performed last, but before the picture deletion processing. It is also possible to similarly handle the case where the encoding / decoding target picture is designated as referable or non-referenceable, and the encoding / decoding target picture is also subjected to picture deletion processing in addition to deletion candidates. . Further, in the case of such a processing order, when there is no free space in the picture memory and all pictures are designated to be referable, the picture to be decoded is made unreferenceable by the header information. If the picture is displayed immediately, the process can proceed to the display process without saving it in the picture memory. Therefore, it is not necessary to perform the memory management process described in the above embodiment. The state before decoding can be used for reference in the decoding process of the next picture. Similarly, when the operation of the decoding device assumed in the encoding device is as described above, the memory management in the encoding device does not need to perform the processing described in the above embodiment. The data in the picture memory can be used for reference in the encoding process of the next picture as it is before encoding.
[0075]
By encoding and decoding a picture using the method described in the above embodiment, it is possible to secure an area for storing data to be decoded in a picture memory in a decoding device. Since it is not necessary to prepare a new memory area for work, the memory capacity of the decoding device and the encoding device can be reduced, and the configurations of the decoding device and the encoding device can be easily designed. Further, the possibility that a picture that cannot be referred to due to the use of the method described in the present embodiment is very low in general encoding and decoding, so that the encoding efficiency is reduced. It can be kept to a minimum.
[0076]
(Embodiment 2)
Embodiment 2 of the present invention describes one embodiment of a moving picture coding method and a decoding method when picture memory management by MMCO is performed, and is a block diagram of the coding apparatus shown in FIG. 2 and the flowchart of the encoding device shown in FIGS. 5 and 6, the flowchart of the decoding device shown in FIGS. 10 and 11, and FIGS. 18 to 23. First, an overview of encoding and decoding will be described using the schematic diagram of the picture memory shown, and then details of encoding and decoding based on picture memory management will be described for each path in the flowchart.
[0077]
FIG. 2 is a block diagram of a decoding device according to the second embodiment. The configuration of the decoding device and the flow of processing are almost the same as those of the conventional method described with reference to FIG. The difference from the conventional method is that the writable area determination unit 208 is added and the working picture memory 207 is removed.
[0078]
In the conventional method, after the decoding of the target picture is completed, the picture memory control unit 206 designates the picture in the picture memory 203 as referable and non-referable, and further performs the picture deletion processing. After that, a procedure of storing the target decoded picture in the picture memory 203 was performed. Therefore, during decoding, data cannot be stored in the picture memory 203, so that a working picture memory 207 is prepared separately from the picture memory 203 to perform decoding.
[0079]
On the other hand, in the present embodiment, it is necessary to analyze beforehand how the picture memory is managed before decoding the current picture and to overwrite the pictures stored in the picture memory 203. Is determined by the writable area determination unit 208, and the decoded data generated while decoding is sequentially stored in the picture memory 203 while being overwritten. Therefore, decoding can be performed without the need for the working picture memory 207, so that the memory in the decoding device can be significantly reduced.
[0080]
In particular, the working picture memory is H.264. Since it is not clearly defined by the H.264 standard, it is necessary to set the memory capacity assuming the largest possible picture size to be input, which is a great obstacle to implementation. On the other hand, since the capacity of the picture memory is clearly defined in the standard, the present embodiment that enables all decoding processes to be performed within the capacity simplifies the implementation of the decoding device. It is possible.
[0081]
FIG. 1 is a block diagram of an encoding device according to Embodiment 2. The configuration of the encoding device and the flow of processing are almost the same as those of the conventional method described with reference to FIG. The difference from the conventional method is that the referenceable picture determination unit 112 is added and the working picture memory 111 is removed.
[0082]
In the decoding device according to the present embodiment, since the decoded picture data is stored as it is in the picture memory while being overwritten, it is possible to overwrite when encoding the picture with the coding device. An encoding method is required on the premise that an area is secured in the picture memory. That is, instead of performing coding by simply referring to the picture stored in the picture memory 105 based on the information indicating whether or not the picture can be referred to as in the conventional method, the picture memory management of the decoding apparatus is performed. Assuming a method at the time of encoding, it is necessary to specify which picture is overwritten at the time of decoding, and to perform encoding without referring to the picture at the time of encoding. The referable picture determination unit 112 specifies a picture that is prohibited from being referred to, and when the motion vector detection unit 106 refers to an encoded picture in the picture memory 105, whether the reference is possible. To the motion vector detection unit 106.
[0083]
In the present embodiment, since the picture memory is managed by the MMCO, a management command is generated in the memory management command generation unit 110, and a flag indicating the use of the MMCO is included in the slice header together with the command information. Describe in.
[0084]
FIG. 27 shows an example of a code string generated by the coding device according to the present embodiment. Similarly to the conventional method, the flag mmco_flg specifies that the picture memory is managed using the MMCO. The management command mmco_cmd is command data necessary for specifying an arbitrary picture to be unreferenceable using the MMCO, and is described in a slice header as shown in the figure. On the other hand, as a point different from the conventional method, a flag dpb_flg is described in the picture information area. This flag is a flag indicating that the code string can be decoded only by the picture memory by the method described in the present embodiment when decoding. The flag dpb_flg is not necessarily required for decoding in a decoding device having a conventional configuration, but the decoding device having the configuration described in the present embodiment analyzes the flag. This makes it possible to determine whether decoding is possible only with the picture memory. The flag dpb_flg is described in the picture common information area in the example of FIG. 27, but may be described in the sequence common information area or another information area.
[0085]
Next, details of the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device and the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device will be described with reference to FIGS. An explanation will be given for each of these routes using the flowchart shown in FIG.
[0086]
(Route 1)
FIG. 10 is a flowchart for explaining the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 1 will be described. FIG. 18 is a diagram for explaining how the state of the picture memory in path 1 changes. In the example shown in the figure, six pictures can be stored, and the pictures are stored from left to right in order of decoding. P0, P3, B1, and B2 represent the pictures shown in FIG. 40, and the numbers indicate the display order. used indicates a picture that can be referred to, and unused indicates a picture that cannot be referred to. Also, Current and C indicate data of a decoding target picture.
[0087]
As shown in FIG. 10, first, before starting the decoding of the current picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and makes the picture which cannot be referred to after the decoding process ends. Is stored. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 18A shows the state of the picture memory before the decoding is started. According to this figure, it can be seen that there are two writing areas in the picture memory. According to the path 1, if there is a free area, it is instructed to decode the target picture while storing the data being decoded in that area. That is, the decoding of the current picture is performed while the data being decoded is stored in the free area designated by the writable area determination unit 208 as shown in FIG. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Originally, in FIG. 18C, an unreferable designation and deletion of a picture are performed for an arbitrary picture. However, FIG. 18C shows an example in which designation of unreferenceable is not performed. Has not been deleted. FIG. 18D shows a state of the picture memory after the memory management processing is applied. In the drawing, since pictures in the decoding order from the left are stored from the left, the data is described for convenience as if the data were rearranged from FIG. 18C to FIG. 18D. It does not move data. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all of these processes are completed, a memory state as shown in FIG. 18E is obtained, and is used for reference in decoding the next picture.
[0088]
On the other hand, FIG. 5 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 1 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 18 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0089]
First, before starting the encoding of the target picture, the memory management command generation unit 110 generates a management command to determine a picture to be made unreferenceable after the encoding is completed. Then, in order to determine a picture that can be referred to, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 18A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there are two writing areas in the assumed picture memory. According to the path 1, when there is a free area, it is instructed to encode the target picture while referring to all the referenceable pictures. That is, it is possible to perform encoding using all four referenceable pictures P0, P3, B1, and B2. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. Originally, in FIG. 18 (c), designation of non-reference is performed for an arbitrary picture, but the figure shows an example in which designation of non-reference is not performed. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0090]
(Route 2)
FIG. 10 is a flowchart for explaining the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 2 will be described. FIG. 19 is a diagram for explaining how the state of the picture memory in path 2 changes.
[0091]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 19A shows the state of the picture memory before decoding is started. According to this figure, it can be seen that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. FIG. 19A shows that B1 and B2 are non-referenceable pictures. Next, it is determined whether or not there is any reference-disabled picture having display order information older than the picture disabled by the management command. In this example, as shown in FIG. 19C, the picture scheduled to be made unreferenceable by the management command is P3. According to FIG. 19A, it is understood that the non-referenceable picture having the oldest display order information is B1, the display order information is 1, and is older than the display order information 3 of P3. As a result, according to the path 2, it is instructed to decode the target picture while storing the data being decoded in the area of the unreferenceable picture having the oldest display order information. That is, as shown in FIG. 19B, the current picture is decoded while the data being decoded is stored in the non-referenceable picture area B1 having the oldest display order information designated by the writable area determination unit 208. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although the MMCO can make an arbitrary picture inaccessible, it is already known that the MMCO command is analyzed and P3 is made inaccessible. Therefore, as shown in FIG. 19 (c), it is specified that reference to P3 is disabled. After that, as described in the related art with reference to FIG. 38, the non-referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 19C, P3, B1, and B2 are deletion candidates, but B1 is deleted because B1 has the oldest display order information. However, physically, since the decoding target picture has already been overwritten on B1, it is not necessary to delete data only by deleting management information. FIG. 19D shows the state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the target picture, it is determined whether the target picture can be referred to or cannot be referred to. As shown in FIG. Do. When all these processes are completed, a memory state as shown in FIG. 19E is obtained, and is used for reference in decoding the next picture.
[0092]
On the other hand, FIG. 5 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 2 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 19 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0093]
First, before starting the encoding of the target picture, the memory management command generation unit 110 generates a management command to determine a picture to be made unreferenceable after the encoding is completed. Then, in order to determine a picture that can be referred to, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 19 (a) shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. FIG. 19A shows that B1 and B2 are non-referenceable pictures. Next, it is determined whether or not there is any reference-disabled picture having display order information older than the picture disabled by the management command. In this example, as shown in FIG. 19C, the picture scheduled to be made unreferenceable by the management command is P3. According to FIG. 19A, it is understood that the non-referenceable picture having the oldest display order information is B1, the display order information is 1, and is older than the display order information 3 of P3. As a result, according to the path 2, it is instructed to encode the target picture while referring to all the referenceable pictures. That is, it is possible to perform encoding using all four referable pictures P0, P3, P6, and B4. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, an arbitrary picture can be made unreferenceable, but it has been determined that an MMCO command has already been generated and P3 has been made unreferenceable. Therefore, as shown in FIG. 19 (c), it is specified that reference to P3 is disabled. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0094]
(Route 3)
FIG. 10 is a flowchart for explaining the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 3 will be described. FIG. 20 is a diagram for explaining how the state of the picture memory changes in the path 3.
[0095]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 20A shows the state of the picture memory before the start of decoding, and it can be seen from this figure that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. FIG. 20A shows that B1 and B2 are non-referenceable pictures. Next, it is determined whether or not there is any reference-disabled picture having display order information older than the picture disabled by the management command. In this example, as shown in FIG. 20C, the picture scheduled to be made unreferenceable by the management command is P0. According to FIG. 20A, the non-referenceable picture having the oldest display order information is B1, the display order information is 1, and it is understood that the display order information 0 of P0 is older. As a result, according to the path 3, it is instructed to perform decoding while saving the picture in the area of the picture which is to be made unreferenceable by the management command. In other words, as shown in FIG. 20B, the decoding of the current picture is performed while the data being decoded is stored in the area P0 of the picture to be made unreferenceable by the management command designated by the writable area determination unit 208. Do. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although the MMCO can make an arbitrary picture inaccessible, it is already known that the MMCO command is analyzed and P0 is made inaccessible. Therefore, as shown in FIG. 20 (c), it is specified that reference to P0 is disabled. After that, as described in the related art with reference to FIG. 38, the non-referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 20C, P0, B1, and B2 are deletion candidates, but P0 is deleted because P0 has the oldest display order information. However, physically, the picture to be decoded has already been overwritten on P0, so there is no need to delete data only by deleting management information. FIG. 20D shows the state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Do. When all these processes are completed, a memory state as shown in FIG. 20E is obtained, and is used for reference in decoding the next picture.
[0096]
On the other hand, FIG. 5 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 3 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 20 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0097]
First, before starting the encoding of the target picture, the memory management command generation unit 110 generates a management command to determine a picture to be made unreferenceable after the encoding is completed. Then, in order to determine a picture that can be referred to, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 20A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. FIG. 20A shows that B1 and B2 are non-referenceable pictures. Next, it is determined whether or not there is any reference-disabled picture having display order information older than the picture disabled by the management command. In this example, as shown in FIG. 20C, the picture scheduled to be made unreferenceable by the management command is P0. According to FIG. 20A, the non-referenceable picture having the oldest display order information is B1, the display order information is 1, and it is understood that the display order information 0 of P0 is older. As a result, according to the path 3, it is instructed to perform encoding without referring to a picture that is to be made unreferenceable by the management command. That is, encoding is performed by referring to only three of the four referenceable pictures P0, P3, P6, and B4, excluding P0, which is to be made unreferenceable. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible, but it has been determined that an MMCO command has already been generated and P0 is made inaccessible. Therefore, as shown in FIG. 20 (c), it is specified that reference to P0 is disabled. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0098]
(Route 4)
FIG. 10 is a flowchart for explaining the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 3 will be described. FIG. 21 is a diagram for explaining a state in which the state of the picture memory in path 4 changes.
[0099]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 21A shows the state of the picture memory before the decoding is started, and it can be seen from this figure that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. According to FIG. 21A, it is understood that there is no non-referenceable picture. Next, it is determined whether there is any picture that cannot be referred to by the management command. According to the MMCO command analyzed in advance, in this example, it is determined that B2 cannot be referred to as shown in FIG. 21C. As a result, according to the path 4, it is instructed to perform the decoding while saving the picture in the area of the picture which is to be made unreferenceable by the management command. That is, as shown in FIG. 21B, the decoding of the current picture is performed while the data being decoded is stored in the area B2 of the picture to be made unreferenceable by the management command designated by the writable area determination unit 208. Do. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although the MMCO can make an arbitrary picture inaccessible, it is already known that the MMCO command is analyzed and B2 is made inaccessible. Therefore, as shown in FIG. 21C, it is specified (marked) that B2 cannot be referenced. After that, as described in the related art with reference to FIG. 38, the non-referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 21C, only B2 is a deletion candidate, so B2 is deleted. However, since the picture to be decoded has already been physically overwritten on B2, there is no need to delete data only by deleting management information. FIG. 21D shows the state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the target picture, it is determined whether the target picture can be referred to or cannot be referred to. As shown in FIG. Do. When all of these processes are completed, a memory state as shown in FIG. 21E is obtained, and is used for reference in decoding the next picture.
[0100]
On the other hand, FIG. 5 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 4 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 21 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0101]
First, before starting the encoding of the target picture, the memory management command generation unit 110 generates a management command to determine a picture to be made unreferenceable after the encoding is completed. Then, in order to determine a picture that can be referred to, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 21A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. According to FIG. 21A, it is understood that there is no non-referenceable picture. Next, it is determined whether or not there is a picture to be made unreferenceable by the management command. In this example, it is determined by a previously generated MMCO command that B2 cannot be referenced as shown in FIG. 21C. As a result, according to the path 4, it is instructed to perform encoding without referring to a picture that is to be made unreferenceable by the management command. That is, the encoding is performed by referring to only five of the six referenceable pictures P0, P3, B2, P6, B4, and B5, excluding B2, which is to be made unreferenceable. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible, but it has been determined that an MMCO command has already been generated and B2 has been made inaccessible. Therefore, as shown in FIG. 21 (c), it is specified that the reference to B2 is disabled. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0102]
(Route 5)
FIG. 10 is a flowchart for explaining the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 3 will be described. FIG. 22 is a diagram for explaining how the state of the picture memory in path 5 changes.
[0103]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 22A shows the state of the picture memory before the decoding is started, and it can be seen from this figure that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. According to FIG. 22A, it can be seen that there is no non-referenceable picture. Next, it is determined whether there is any picture that cannot be referred to by the management command. In this example, it is determined by a previously analyzed MMCO command that no picture is to be made unreferenceable as shown in FIG. As a result, according to the path 5, it is instructed to perform the decoding while storing in the area of the referenceable picture having the oldest display order information. That is, the decoding of the target picture is performed while the data being decoded is stored in the area P0 of the referenceable picture having the oldest display order information designated by the writable area determination unit 208 as shown in FIG. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although it is possible to make an arbitrary picture inaccessible in the MMCO, it has been found that the analysis of the command of the MMCO has already been performed and no picture can be made inaccessible. Therefore, as described in the related art with reference to FIG. 38, the referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 22C, the referenceable picture having the oldest display order information is P0, and P0 is deleted. However, physically, the picture to be decoded has already been overwritten on P0, so there is no need to delete data only by deleting management information. FIG. 22D shows a state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all of these processes are completed, a memory state as shown in FIG. 22E is obtained, and is used for reference in decoding the next picture.
[0104]
On the other hand, FIG. 5 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 5 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 22 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0105]
First, before starting the encoding of the target picture, the memory management command generation unit 110 generates a management command to determine a picture to be made unreferenceable after the encoding is completed. Then, in order to determine a picture that can be referred to, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 22A shows the state of the assumed picture memory before the start of the encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. According to FIG. 22A, it can be seen that there is no non-referenceable picture. Next, it is determined whether or not there is a picture to be made unreferenceable by the management command. In this example, it is determined by a previously generated MMCO command that none of the pictures is disabled from being referenced as shown in FIG. As a result, according to the path 5, it is instructed to perform the encoding without referring to the referenceable picture having the oldest display order information. That is, encoding is performed by referring to only five of the six referenceable pictures P0, P3, B2, P6, B4, and B5 except P0 having the oldest display order information. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible, but it has been determined that an MMCO command has already been generated and no picture can be made inaccessible. However, as described in the related art with reference to FIG. 38, since the referenceable picture having the oldest display order information is deleted in the decoding device, the oldest display order information is also deleted in the encoding device. It is necessary to delete the P0 possessed or to make it inaccessible. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0106]
(Route 2a)
FIG. 11 is a flowchart for explaining operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. The only difference from the operation of the flowchart of FIG. 10 is that the picture memory control unit 206 has a function of determining whether a decoded picture in the picture memory has already been displayed. Therefore, the route 1, the route 3, the route 4, and the route 5 are completely the same as the embodiment described with reference to FIG. Here, the path 2a will be described. FIG. 23 is a diagram for explaining how the state of the picture memory changes in the path 2a.
[0107]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 23A shows the state of the picture memory before decoding is started, and it can be seen from this figure that there is no free area in the picture memory. If there is no free area, it is next determined whether there is a non-referenceable picture in the picture memory. FIG. 23A shows that B1 and B2 are non-referenceable pictures. Next, it is determined whether or not there is display order information that is older than the picture that is made unreferenceable by the management command or there is an already displayed picture that cannot be referenced. In this example, as shown in FIG. 23C, the picture scheduled to be made unreferenceable by the management command is set to P0. According to FIG. 23A, it is understood that the non-referenceable picture having the oldest display order information is B1, the display order information is 1, and 0 of the display order information of P0 is older. However, P0, P3, B1, and B2 are displayed pictures, and B1 and B2 are displayed pictures that cannot be referenced. As a result, according to the path 2a, it is instructed to decode the target picture while storing the data being decoded in the area of the non-referenceable picture having the oldest display order information. That is, the current picture is decoded while the data being decoded is stored in the area B1 of the non-referenceable picture having the oldest display order information designated by the writable area determination unit 208 as shown in FIG. . Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although the MMCO can make an arbitrary picture inaccessible, it is already known that the MMCO command is analyzed and P0 is made inaccessible. Therefore, as shown in FIG. 23 (c), it is designated that reference to P0 is disabled. After that, as described in the related art with reference to FIG. 38, the non-referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 23C, P0, B1, and B2 are deletion candidates, but P0 is deleted because P0 has the oldest display order information. Also, since B1 has already been overwritten by the picture to be decoded, the originally stored data as B1 has already been lost. Therefore, B1 is removed from the state of the picture memory after the memory management processing shown in FIG. 23D is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to, and these designations are made for the current picture as shown in FIG. Do. When all these processes are completed, a memory state as shown in FIG. 23E is obtained, and is used for reference in decoding the next picture.
[0108]
On the other hand, FIG. 6 is a flowchart for explaining the operations of the picture memory control unit 109 and the referable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. The only difference from the operation of the flowchart of FIG. 5 is that the picture memory control unit 109 has a function of determining whether a decoded picture in the assumed picture memory has already been displayed. Therefore, the route 1, the route 3, the route 4, and the route 5 are completely the same as the embodiment described with reference to FIG. Here, the path 2a will be described. FIG. 23 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0109]
First, before starting the encoding of the target picture, the memory management command generation unit 110 generates a management command to determine a picture to be made unreferenceable after the encoding is completed. Then, in order to determine a picture that can be referred to, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 23A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not there is a non-referenceable picture in the assumed picture memory. FIG. 23A shows that B1 and B2 are non-referenceable pictures. Next, it is determined whether or not there is display order information that is older than the picture that is made unreferenceable by the management command or there is an already displayed picture that cannot be referenced. In this example, as shown in FIG. 23C, the picture scheduled to be made unreferenceable by the management command is set to P0. According to FIG. 23A, it is understood that the non-referenceable picture having the oldest display order information is B1, the display order information is 1, and 0 of the display order information of P0 is older. However, P0, P3, B1, and B2 are displayed pictures, and B1 and B2 are displayed pictures that cannot be referenced. As a result, according to the path 2a, it is instructed to encode the target picture while referring to all the referenceable pictures. That is, it is possible to perform encoding using all four referable pictures P0, P3, P6, and B4. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible, but it has been determined that an MMCO command has already been generated and P0 is made inaccessible. Therefore, as shown in FIG. 23 (c), it is designated that reference to P0 is disabled. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0110]
In the embodiment of the present invention, in any of the descriptions of the paths, the work of designating the reference and non-reference of the picture to be encoded and decoded is performed last, but before the picture deletion processing. It is also possible to similarly handle the case where the encoding / decoding target picture is designated as referable or non-referenceable, and the encoding / decoding target picture is also subjected to picture deletion processing in addition to deletion candidates. . Further, in the case of such a processing order, when there is no free space in the picture memory and all pictures are designated to be referable, the picture to be decoded is made unreferenceable by the header information. If the picture is displayed immediately, the process can proceed to the display process without saving it in the picture memory. Therefore, it is not necessary to perform the memory management process described in the above embodiment. The state before decoding can be used for reference in the decoding process of the next picture. Similarly, when the operation of the decoding device assumed in the encoding device is as described above, the memory management in the encoding device does not need to perform the processing described in the above embodiment. The data in the picture memory can be used for reference in the encoding process of the next picture as it is before encoding.
[0111]
In the embodiment of the present invention, regardless of the state of the assumed picture memory by the referenceable picture determination unit 112 in the encoding device, the decoding target is stored in the picture memory of the assumed decoding device. However, the memory management command generation unit 110 cannot always refer to the picture deleted from the picture memory of the assumed decoding device after the decoding of the current picture is completed by the memory management command generation unit 110. By generating an MMCO command as follows, the same purpose can be realized. In that case, the condition is that a command is generated so as to pass only the route 1, the route 2, and the route 2a in FIGS. 5 and 6, and it is necessary to satisfy any of the following.
{Circle around (1)} The picture memory of the decoding apparatus has a free area for one picture or more.
{Circle around (2)} When there is no free space in the picture memory of the decoding device, there is one or more non-referenceable pictures, and a command to disable reference to referenceable pictures having display order information older than all the non-referenceable pictures is issued. Do not generate.
[0112]
{Circle around (3)} When the decoding device can determine whether or not the decoded picture in the picture memory has been displayed, if there is no free space in the picture memory of the decoding device, an unreferenceable picture is displayed. When one or more pictures exist and not all the non-referenceable pictures have been displayed, a command for disabling referenceable pictures having display order information older than all the non-referenceable pictures is not generated.
[0113]
By using the above method, the encoding device can always perform encoding with reference to all the referenceable pictures, and can simplify the process of determining the referenceable picture.
[0114]
By encoding and decoding a picture using the method described in the above embodiment, it is possible to secure an area for storing data to be decoded in a picture memory in a decoding device. Since it is not necessary to prepare a new memory area for work, the memory capacity of the decoding device and the encoding device can be reduced, and the configurations of the decoding device and the encoding device can be easily designed. Further, the possibility that a picture that cannot be referred to due to the use of the method described in the present embodiment is very low in general encoding and decoding, so that the encoding efficiency is reduced. It can be kept to a minimum.
[0115]
(Embodiment 3)
The third embodiment of the present invention describes a moving picture encoding method and a decoding method when picture memory management by MMCO is performed, which is different from the first embodiment. The block diagram of the decoding device, the block diagram of the decoding device shown in FIG. 2, the flowchart of the encoding device shown in FIG. 7, and the flowchart of the decoding device shown in FIG. 12, and FIGS. First, an overview of encoding and decoding will be described using the schematic diagram of the picture memory shown, and then details of encoding and decoding based on picture memory management will be described for each path in the flowchart.
[0116]
FIG. 2 is a block diagram of a decoding device according to the third embodiment. The configuration of the decoding device and the flow of processing are almost the same as those of the conventional method described with reference to FIG. The difference from the conventional method is that the writable area determination unit 208 is added and the working picture memory 207 is removed.
[0117]
In the conventional method, after the decoding of the target picture is completed, the picture memory control unit 206 designates the picture in the picture memory 203 as referable and non-referable, and further performs the picture deletion processing. After that, a procedure of storing the target decoded picture in the picture memory 203 was performed. Therefore, during decoding, data cannot be stored in the picture memory 203, so that a working picture memory 207 is prepared separately from the picture memory 203 to perform decoding.
[0118]
On the other hand, in the present embodiment, it is necessary to analyze beforehand how the picture memory is managed before decoding the current picture and to overwrite the pictures stored in the picture memory 203. Is determined by the writable area determination unit 208, and the decoded data generated while decoding is sequentially stored in the picture memory 203 while being overwritten. Therefore, decoding can be performed without the need for the working picture memory 207, so that the memory in the decoding device can be significantly reduced.
[0119]
In particular, the working picture memory is H.264. Since it is not clearly defined by the H.264 standard, it is necessary to set the memory capacity assuming the largest possible picture size to be input, which is a great obstacle to implementation. On the other hand, since the capacity of the picture memory is clearly defined in the standard, the present embodiment that enables all decoding processes to be performed within the capacity simplifies the implementation of the decoding device. It is possible.
[0120]
FIG. 1 is a block diagram of an encoding device according to Embodiment 3. The configuration of the encoding device and the flow of processing are almost the same as those of the conventional method described with reference to FIG. The difference from the conventional method is that the referenceable picture determination unit 112 is added and the working picture memory 111 is removed.
[0121]
In the decoding device according to the present embodiment, since the decoded picture data is stored as it is in the picture memory while being overwritten, it is possible to overwrite when encoding the picture with the coding device. An encoding method is required on the premise that an area is secured in the picture memory. That is, instead of performing coding by simply referring to the picture stored in the picture memory 105 based on the information indicating whether or not the picture can be referred to as in the conventional method, the picture memory management of the decoding apparatus is performed. Assuming a method at the time of encoding, it is necessary to specify which picture is overwritten at the time of decoding, and to perform encoding without referring to the picture at the time of encoding. The referable picture determination unit 112 specifies a picture that is prohibited from being referred to, and when the motion vector detection unit 106 refers to an encoded picture in the picture memory 105, whether the reference is possible. To the motion vector detection unit 106.
[0122]
In the present embodiment, since the picture memory is managed by the MMCO, a management command is generated in the memory management command generation unit 110, and a flag indicating the use of the MMCO is included in the slice header together with the command information. Describe in.
[0123]
FIG. 27 shows an example of a code string generated by the coding device according to the present embodiment. Similarly to the conventional method, the flag mmco_flg specifies that the picture memory is managed using the MMCO. The management command mmco_cmd is command data necessary for specifying an arbitrary picture to be unreferenceable using the MMCO, and is described in a slice header as shown in the figure. On the other hand, as a point different from the conventional method, a flag dpb_flg is described in the picture information area. This flag is a flag indicating that the code string can be decoded only by the picture memory by the method described in the present embodiment when decoding. The flag dpb_flg is not necessarily required for decoding in a decoding device having a conventional configuration, but the decoding device having the configuration described in the present embodiment analyzes the flag. This makes it possible to determine whether decoding is possible only with the picture memory. The flag dpb_flg is described in the picture common information area in the example of FIG. 27, but may be described in the sequence common information area or another information area.
[0124]
Next, details of the operations of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device and the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device will be described with reference to FIGS. An explanation will be given for each of these routes using the flowchart shown in FIG.
[0125]
(Route 1)
FIG. 12 is a flowchart for explaining the operation of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 1 will be described. FIG. 24 is a diagram for explaining how the state of the picture memory in path 1 changes. In the example shown in the figure, six pictures can be stored, and the pictures are stored from left to right in order of decoding. P0, P3, B1, and B2 represent the pictures shown in FIG. 40, and the numbers indicate the display order. used indicates a picture that can be referred to, and unused indicates a picture that cannot be referred to. Also, Current and C indicate data of a decoding target picture.
[0126]
First, before starting decoding of the current picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 24A shows the state of the picture memory before decoding is started. According to this figure, it can be seen that there are two writing areas in the picture memory. According to the path 1, if there is a free area, it is instructed to decode the target picture while storing the data being decoded in that area. That is, the decoding of the current picture is performed while the data being decoded is stored in the free area designated by the writable area determination unit 208 as shown in FIG. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Originally, in FIG. 24 (c), designation of non-reference to an arbitrary picture and deletion of the picture are performed. However, since there is a free area in the picture memory, the picture deletion processing is not performed. FIG. 24D shows a state of the picture memory after the memory management processing is applied. In the drawing, since pictures in the decoding order from the left are stored from the left, the data is described for convenience as if the data were rearranged from FIG. 24 (c) to FIG. 24 (d). It does not move data. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all these processes are completed, a memory state as shown in FIG. 24E is obtained, and is used for reference in decoding the next picture.
[0127]
On the other hand, FIG. 7 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 1 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 24 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0128]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 24A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there are two writing areas in the assumed picture memory. According to the path 1, if there is an empty area, first, the memory management command generation unit 110 arbitrarily determines a picture to be made unreferenceable after encoding is completed, and issues a management command for making the picture unreferenceable. Generate. Next, the current picture is coded while referring to all the referenceable pictures. That is, it is possible to perform encoding using all four referenceable pictures P0, P3, B1, and B2. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible, but it has been determined that an MMCO command has already been generated and B2 has been made inaccessible. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0129]
(Route 2)
FIG. 12 is a flowchart for explaining the operation of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 2 will be described. FIG. 25 is a diagram for explaining how the state of the picture memory changes in the path 2.
[0130]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 25 (a) shows the state of the picture memory before decoding starts, and it can be seen from this figure that there is no free area in the picture memory. As a result, according to the path 2, it is instructed to decode the target picture while storing the data being decoded in the area of the picture having the oldest display order information. That is, as shown in FIG. 25B, the current picture is decoded while the data being decoded is stored in the area P0 of the picture having the oldest display order information designated by the writable area determination unit 208. In the case of the example shown in this figure, the picture having the oldest display order information is a non-referenceable picture. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although it is possible to make an arbitrary picture inaccessible in the MMCO, it has been found that the analysis of the command of the MMCO has already been performed and no picture can be made inaccessible. After that, as described in the related art with reference to FIG. 38, the non-referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 25C, P0 and B2 are deletion candidates, but P0 is deleted because P0 has the oldest display order information. However, physically, the picture to be decoded has already been overwritten on P0, so there is no need to delete data only by deleting management information. FIG. 25D shows a state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all these processes are completed, a memory state as shown in FIG. 25E is obtained, and is used for reference in decoding the next picture.
[0131]
On the other hand, FIG. 7 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 2 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 25 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0132]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 25A shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not the picture having the display order information is inaccessible. According to FIG. 25A, the picture having the most display order information is P0, and it can be seen that P0 is designated as non-referenceable. As a result, according to the path 2, first, the memory management command generation unit 110 arbitrarily determines a picture to be made inaccessible after encoding is completed, and generates a management command to make the picture inaccessible. Next, the current picture is coded while referring to all the referenceable pictures. That is, encoding can be performed using all four referenceable pictures P3, B1, P6, and B4. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible, but it has been determined that an MMCO command has already been generated and no picture can be made inaccessible. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0133]
(Route 3)
FIG. 12 is a flowchart for explaining the operation of the picture memory control unit 206 and the writable area determination unit 208 in the decoding device. Here, the route 2 will be described. FIG. 26 is a diagram for explaining how the state of the picture memory in path 3 changes.
[0134]
Before starting the decoding of the target picture, the writable area determination unit 208 analyzes the MMCO command included in the code string, and stores a picture that cannot be referred to after the decoding process ends. Thereafter, in order to determine the writable area, it is determined whether or not there is a free area in which the decoded picture is not stored in the code string picture memory 203. FIG. 26A shows the state of the picture memory before decoding is started. According to this figure, it can be seen that there is no free area in the picture memory. As a result, according to the path 3, it is instructed to decode the target picture while saving the data being decoded in the area of the picture having the oldest display order information. That is, as shown in FIG. 26B, the current picture is decoded while the data being decoded is stored in the area B1 of the picture having the oldest display order information designated by the writable area determination unit 208. In the case of the example shown in this figure, the picture having the oldest display order information is the referenceable picture. Next, a memory management process is performed by the picture memory control unit 206. In the present embodiment, memory management by MMCO is performed. Although the MMCO can make an arbitrary picture inaccessible, it is already known that the MMCO command is analyzed and B1 is made inaccessible. Therefore, as shown in FIG. 26 (c), it is specified that the reference to B1 is disabled. After that, as described in the related art with reference to FIG. 38, the non-referenceable picture having the oldest display order information is deleted. That is, in the example of FIG. 26C, B1 and B2 are deletion candidates, but B1 is deleted since B1 has the oldest display order information. However, physically, since the decoding target picture has already been overwritten on B1, it is not necessary to delete data only by deleting management information. FIG. 26D shows a state of the picture memory after the memory management processing is applied. Finally, based on the information described in the header area of the current picture, it is determined whether the current picture can be referred to or cannot be referred to. As shown in FIG. Marking). When all these processes are completed, a memory state as shown in FIG. 26E is obtained, and is used for reference in decoding the next picture.
[0135]
On the other hand, FIG. 7 is a flowchart for explaining the operations of the picture memory control unit 109 and the referenceable picture determination unit 112 in the encoding device that generates a code string that can be decoded as described above. Here, the route 2 will be described. The encoding device according to the present embodiment performs encoding while assuming the memory management of the corresponding decoding device. FIG. 26 is a diagram for explaining how the state of the picture memory in the decoding device assumed at this time changes.
[0136]
First, before starting the encoding of the current picture, the referenceable picture determination unit 112 determines whether or not there is a free area in the picture memory of the assumed decoding device. FIG. 26 (a) shows the state of the assumed picture memory before the start of encoding. According to this figure, it can be seen that there is no free area in the assumed picture memory. If there is no free area, it is next determined whether or not the picture having the display order information is inaccessible. According to FIG. 26A, the picture having the most display order information is B1, and it can be seen that B1 is designated to be referable. As a result, according to the path 3, first, the memory management command generation unit 110 selects a picture whose display order information is the oldest as a picture to be made inaccessible after encoding is completed, and performs management for making the picture inaccessible. Generate commands. According to FIG. 26 (c), the picture with the oldest display order information is B1, and a command to disable B1 is generated. Next, encoding is performed without referring to the picture having the oldest display order information. That is, encoding is performed by referring to only four pictures except for the picture B1 with the oldest display order information among the five referable pictures P3, B1, P6, B4, and B5. Next, a memory management process is performed by the picture memory control unit 109. In the present embodiment, memory management by MMCO is performed. In the MMCO, it is possible to make an arbitrary picture inaccessible. However, it has been determined that an MMCO command has already been generated and B1 has been made inaccessible. In the encoding device, the method of deleting the picture in the picture memory is arbitrary, and the same method as that of the decoding device or another method can be used. Finally, it is determined whether or not the encoding target picture can be referred to, and the picture is stored in the picture memory as needed.
[0137]
In the embodiment of the present invention, in any of the descriptions of the paths, the work of designating the reference and non-reference of the picture to be encoded and decoded is performed last, but before the picture deletion processing. It is also possible to similarly handle the case where the encoding / decoding target picture is designated as referable or non-referenceable, and the encoding / decoding target picture is also subjected to picture deletion processing in addition to deletion candidates. . Further, in the case of such a processing order, when there is no free space in the picture memory and all pictures are designated to be referable, the picture to be decoded is made unreferenceable by the header information. If the picture is displayed immediately, the process can proceed to the display process without saving it in the picture memory. Therefore, it is not necessary to perform the memory management process described in the above embodiment. The state before decoding can be used for reference in the decoding process of the next picture. Similarly, when the operation of the decoding device assumed in the encoding device is as described above, the memory management in the encoding device does not need to perform the processing described in the above embodiment. The data in the picture memory can be used for reference in the encoding process of the next picture as it is before encoding.
[0138]
By encoding and decoding a picture using the method described in the above embodiment, it is possible to secure an area for storing data to be decoded in a picture memory in a decoding device. Since it is not necessary to prepare a new memory area for work, the memory capacity of the decoding device and the encoding device can be reduced, and the configurations of the decoding device and the encoding device can be easily designed. Further, the possibility that a picture that cannot be referred to due to the use of the method described in the present embodiment is very low in general encoding and decoding, so that the encoding efficiency is reduced. It can be kept to a minimum. In comparison with the second embodiment, the possibility of occurrence of a picture that cannot be referred to is not high, but the determination process in the memory management is simplified, and the efficiency of the process can be improved.
[0139]
In the embodiment of the present invention, regardless of the state of the assumed picture memory by the referenceable picture determination unit 112 in the encoding device, the decoding target is stored in the picture memory of the assumed decoding device. However, the memory management command generation unit 110 cannot always refer to the picture deleted from the picture memory of the assumed decoding device after the decoding of the current picture is completed by the memory management command generation unit 110. By generating an MMCO command as follows, the same purpose can be realized. In this case, a condition is to generate a command to pass only the route 1 and the route 2 in FIG. 7, and it is necessary to satisfy any of the following.
[0140]
{Circle around (1)} The picture memory of the decoding apparatus has a free area for one picture or more.
{Circle over (2)} When there is no free space in the picture memory of the decoding device, a command in which the picture having the oldest display order information cannot be referred to when decoding the picture following the target picture is used. To generate.
[0141]
By using the above method, the encoding device can always perform encoding with reference to all the referenceable pictures, and can simplify the process of determining the referenceable picture.
[0142]
(Embodiment 4)
FIG. 28 illustrates a case where the image encoding method used in the image encoding device according to the above embodiment and the image decoding method used in the image decoding device are implemented by a computer system using a program stored in a flexible disk. FIG.
[0143]
FIG. 28B shows the appearance, cross-sectional structure, and flexible disk of the flexible disk as viewed from the front, and FIG. 28A shows an example of the physical format of the flexible disk which is a recording medium body. The flexible disk FD is built in the case F, and a plurality of tracks Tr are formed concentrically from the outer circumference toward the inner circumference on the surface of the disk, and each track is divided into 16 sectors Se in an angular direction. ing. Therefore, in the flexible disk storing the program, an image encoding method and an image decoding method as the program are recorded in an area allocated on the flexible disk FD.
[0144]
FIG. 28C shows a configuration for recording and reproducing the program on the flexible disk FD. When the above program is recorded on the flexible disk FD, the image encoding method and the image decoding method as the above program are written from the computer system Cs via the flexible disk drive FDD. When the image encoding method and the image decoding method are constructed in a computer system using a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.
[0145]
In the above description, the description has been made using a flexible disk as a recording medium. However, the same description can be made using an optical disk. Further, the recording medium is not limited to this, and can be similarly implemented as long as the program can be recorded, such as a CD-ROM, a memory card, a ROM cassette, and the like.
[0146]
Further, here, application examples of the image encoding method and the image decoding method described in the above embodiment and a system using the same will be described.
FIG. 29 is a block diagram illustrating an overall configuration of a content supply system ex100 that realizes a content distribution service. A communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed wireless stations, are installed in each cell.
[0147]
The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, and a camera on the Internet ex101 via an Internet service provider ex102 and a telephone network ex104, and base stations ex107 to ex110. Each device such as the mobile phone ex115 is connected.
[0148]
However, the content supply system ex100 is not limited to the combination as shown in FIG. 29, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.
[0149]
The camera ex113 is a device such as a digital video camera capable of shooting moving images. In addition, a mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access mobile phone system, or a GSM communication system). Or PHS (Personal Handyphone System) or the like.
[0150]
The streaming server ex103 is connected from the camera ex113 to the base station ex109 and the telephone network ex104, and enables live distribution and the like based on encoded data transmitted by the user using the camera ex113. The encoding process of the photographed data may be performed by the camera ex113, or may be performed by a server or the like that performs the data transmission process. Also, moving image data captured by the camera 116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device such as a digital camera that can shoot still images and moving images. In this case, encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed by the LSI ex117 of the computer ex111 and the camera ex116. It should be noted that the image encoding / decoding software may be incorporated in any storage medium (a CD-ROM, a flexible disk, a hard disk, or the like) that is a recording medium readable by the computer ex111 or the like. Further, the moving image data may be transmitted by the mobile phone with camera ex115. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.
[0151]
In the content supply system ex100, the content (for example, a video image of a live music) captured by the user with the camera ex113, the camera ex116, and the like is encoded and transmitted to the streaming server ex103 as in the above-described embodiment. On the other hand, the streaming server ex103 stream-distributes the content data to the requesting client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like, which can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and further, realizes personal broadcast by receiving, decoding, and reproducing the data in real time at the client. It is a system that becomes possible.
[0152]
The encoding and decoding of each device constituting this system may be performed using the image encoding method or the image decoding method described in each of the above embodiments.
A mobile phone will be described as an example.
[0153]
FIG. 31 is a diagram illustrating the mobile phone ex115 that uses the image encoding method and the image decoding method described in the above embodiment. The mobile phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera unit ex203 capable of taking a picture such as a CCD camera, a still image, a picture taken by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display for displaying data obtained by decoding a received video or the like, a main unit including operation keys ex204, an audio output unit ex208 such as a speaker for outputting audio, and audio input. Input unit ex205 such as a microphone for storing encoded or decoded data, such as data of captured moving images or still images, received mail data, moving image data or still image data, etc. Of recording media ex207 to mobile phone ex115 And a slot portion ex206 to ability. The recording medium ex207 stores a flash memory device, which is a kind of electrically erasable and programmable read only memory (EEPROM), which is a nonvolatile memory that can be electrically rewritten and erased, in a plastic case such as an SD card.
[0154]
Further, the mobile phone ex115 will be described with reference to FIG. The mobile phone ex115 is provided with a power supply circuit unit ex310, an operation input control unit ex304, an image encoding unit, and a main control unit ex311 which controls the respective units of a main body unit including a display unit ex202 and operation keys ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via the synchronous bus ex313. Connected to each other.
[0155]
When the end of the call and the power key are turned on by a user operation, the power supply circuit unit ex310 supplies power to each unit from the battery pack to activate the digital cellular phone with camera ex115 in an operable state. .
[0156]
The mobile phone ex115 converts a sound signal collected by the sound input unit ex205 into digital sound data by the sound processing unit ex305 in the voice call mode based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. This is spread-spectrum-processed by a modulation / demodulation circuit unit ex306, subjected to digital-analog conversion processing and frequency conversion processing by a transmission / reception circuit unit ex301, and then transmitted via an antenna ex201. The mobile phone ex115 amplifies the received signal received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-to-digital conversion processing, performs spectrum despreading processing in the modulation / demodulation circuit unit ex306, and performs analog voice decoding in the voice processing unit ex305. After being converted into a signal, the signal is output via the audio output unit ex208.
[0157]
Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital / analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and transmits the data to the base station ex110 via the antenna ex201.
[0158]
When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.
[0159]
The image encoding unit ex312 includes the image encoding device described in the present invention, and uses the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The image data is converted into encoded image data by compression encoding, and is transmitted to the demultiplexing unit ex308. At this time, the mobile phone ex115 simultaneously transmits the audio collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.
[0160]
The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and multiplexes the resulting multiplexed data into a modulation / demodulation circuit unit. The signal is subjected to spread spectrum processing in ex306 and subjected to digital-analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmitted via the antenna ex201.
[0161]
When receiving data of a moving image file linked to a homepage or the like in the data communication mode, the modulation / demodulation circuit unit ex306 performs spectrum despread processing on a reception signal received from the base station ex110 via the antenna ex201, and obtains the resulting multiplexed signal. The demultiplexed data is sent to the demultiplexing unit ex308.
[0162]
Further, to decode the multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data to form an encoded bit stream of image data and an encoded bit stream of audio data. And supplies the encoded image data to the image decoding unit ex309 and the audio data to the audio processing unit ex305 via the synchronous bus ex313.
[0163]
Next, the image decoding unit ex309 is configured to include the image decoding device described in the present invention, and converts a coded bit stream of image data into a decoding method corresponding to the coding method described in the above embodiment. To generate reproduced moving image data and supply it to the display unit ex202 via the LCD control unit ex302, whereby, for example, moving image data included in a moving image file linked to a homepage is displayed. . At this time, at the same time, the audio processing unit ex305 converts the audio data into an analog audio signal, and then supplies the analog audio signal to the audio output unit ex208, thereby reproducing, for example, the audio data included in the moving image file linked to the homepage. You.
[0164]
It should be noted that the present invention is not limited to the example of the above-mentioned system, and digital broadcasting by satellite and terrestrial broadcasting has recently become a topic. As shown in FIG. Any of the decoding devices can be incorporated. Specifically, at the broadcasting station ex409, an encoded bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. The broadcasting satellite ex410 receiving this transmits a radio wave for broadcasting, receives this radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, and transmits the radio wave to a television (receiver) ex401 or a set-top box (STB) ex407 or the like. The device decodes the encoded bit stream and reproduces it. In addition, the image decoding device described in the above embodiment can be mounted on a playback device ex403 that reads and decodes an encoded bit stream recorded on a storage medium ex402 such as a CD or DVD that is a recording medium. is there. In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration is also conceivable in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on a monitor ex408 of the television. At this time, the image decoding device may be incorporated in the television instead of the set-top box. In addition, a car ex412 having an antenna ex411 can receive a signal from the satellite ex410 or a base station ex107 or the like, and can reproduce a moving image on a display device such as a car navigation ex413 included in the car ex412.
[0165]
Further, an image signal can be encoded by the image encoding device described in the above embodiment and recorded on a recording medium. Specific examples include a recorder ex420 such as a DVD recorder that records an image signal on a DVD disc ex421 and a disc recorder that records on a hard disk. Furthermore, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.
[0166]
The configuration of the car navigation system ex413 may be, for example, a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 from the configuration illustrated in FIG. 30, and the same applies to the computer ex111 and the television (receiver). ) Ex401 and the like are also considered.
[0167]
In addition, terminals such as the mobile phone ex114 and the like have three mounting formats, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.
[0168]
As described above, the image encoding method or the image decoding method described in the above embodiment can be used for any of the devices and systems described above, and by doing so, the effects described in the above embodiment can be obtained. Obtainable.
[0169]
【The invention's effect】
As described above, in the moving picture coding apparatus and the decoding apparatus according to the present invention, it is possible to secure an area for storing data to be decoded in the picture memory of the decoding apparatus, and a new work area is provided. Since there is no need to prepare a memory area, the memory capacity of the decoding device and the encoding device can be reduced, and the configurations of the decoding device and the encoding device can be easily designed. Further, the possibility that a picture that cannot be referred to due to the use of the method described in the present embodiment is very low in general encoding and decoding, so that the encoding efficiency is reduced. It can be kept to a minimum.
[Brief description of the drawings]
FIG. 1 is a block diagram for describing an encoding operation according to a first embodiment, a second embodiment, and a third embodiment of the present invention.
FIG. 2 is a block diagram for explaining a decoding operation according to Embodiments 1, 2, and 3 of the present invention.
FIG. 3 is a flowchart illustrating a flow of a memory management process of the encoding device according to the first embodiment of the present invention.
FIG. 4 is a flowchart for explaining a flow of a memory management process of another encoding device according to the first embodiment of the present invention.
FIG. 5 is a flowchart illustrating a flow of a memory management process of an encoding device according to Embodiment 2 of the present invention.
FIG. 6 is a flowchart for explaining a flow of a memory management process of another encoding device according to Embodiment 2 of the present invention.
FIG. 7 is a flowchart illustrating a flow of a memory management process of an encoding device according to Embodiment 3 of the present invention.
FIG. 8 is a flowchart illustrating a flow of a memory management process of the decoding device according to the first embodiment of the present invention.
FIG. 9 is a flowchart for describing a flow of a memory management process of another decoding device according to the first embodiment of the present invention.
FIG. 10 is a flowchart illustrating a flow of a memory management process of the decoding device according to the second embodiment of the present invention.
FIG. 11 is a flowchart for explaining a flow of a memory management process of another decoding device according to the second embodiment of the present invention.
FIG. 12 is a flowchart illustrating a flow of a memory management process of the decoding device according to the third embodiment of the present invention.
FIG. 13 is a conceptual diagram of a picture memory for describing path 1 of a flow of a memory management process of the decoding device according to the first embodiment of the present invention.
FIG. 14 is a conceptual diagram of a picture memory for describing a path 2 of a flow of a memory management process of the decoding device according to the first embodiment of the present invention.
FIG. 15 is a conceptual diagram of a picture memory for describing a path 3 of a flow of a memory management process of the decoding device according to the first embodiment of the present invention.
FIG. 16 is a conceptual diagram of a picture memory for describing a path 4 of a flow of a memory management process of the decoding device according to the first embodiment of the present invention.
FIG. 17 is a conceptual diagram of a picture memory for describing a path 2a of a flow of a memory management process of another decoding device according to the first embodiment of the present invention.
FIG. 18 is a conceptual diagram of a picture memory for describing a path 1 of a flow of a memory management process of the decoding device according to the second embodiment of the present invention.
FIG. 19 is a conceptual diagram of a picture memory for describing a path 2 of a flow of a memory management process of the decoding device according to the second embodiment of the present invention.
FIG. 20 is a conceptual diagram of a picture memory for describing a path 3 of a flow of a memory management process of the decoding device according to the second embodiment of the present invention.
FIG. 21 is a conceptual diagram of a picture memory for describing a path 4 of a flow of a memory management process of the decoding device according to the second embodiment of the present invention.
FIG. 22 is a conceptual diagram of a picture memory for describing a path 5 of a flow of a memory management process of the decoding device according to the second embodiment of the present invention.
FIG. 23 is a conceptual diagram of a picture memory for describing a path 2a of a flow of a memory management process of another decoding device according to the second embodiment of the present invention.
FIG. 24 is a conceptual diagram of a picture memory for describing path 1 of a flow of a memory management process of the decoding device according to the third embodiment of the present invention.
FIG. 25 is a conceptual diagram of a picture memory for describing path 2 of a flow of a memory management process of the decoding device according to the third embodiment of the present invention.
FIG. 26 is a conceptual diagram of a picture memory for describing a path 3 of a flow of a memory management process of the decoding device according to the third embodiment of the present invention.
FIG. 27 is a schematic diagram for explaining a configuration of a code string.
FIG. 28 is an explanatory diagram of a recording medium for storing a program for realizing a moving image encoding method and a moving image decoding method according to each of the embodiments by a computer system.
FIG. 29 is a block diagram illustrating an overall configuration of a content supply system.
FIG. 30 is an external view of a mobile phone.
FIG. 31 is a block diagram illustrating a configuration of a mobile phone.
FIG. 32 is a diagram illustrating an example of a digital broadcasting system.
FIG. 33 is a block diagram for explaining a conventional encoding operation.
FIG. 34 is a block diagram for explaining a conventional decoding operation.
FIG. 35 is a flowchart illustrating a flow of a memory management process of the conventional encoding device.
FIG. 36 is a flowchart for explaining a flow of a memory management process of another conventional encoding device.
FIG. 37 is a flowchart illustrating a flow of a memory management process of the conventional decoding device.
FIG. 38 is a flowchart illustrating a flow of a memory management process of another conventional decoding device.
FIG. 39 is a schematic diagram for explaining a configuration of a conventional code string.
FIG. 40 is a conceptual diagram for describing a reference relationship between pictures.
[Explanation of symbols]
101 picture memory
102 prediction residual encoder
103 Code string generation unit
104 prediction residual decoding unit
105 picture memory
106 motion vector detection unit
107 motion compensation coding unit
108 Motion vector storage unit
109 Picture memory control unit
110 memory management command generator
111 Working picture memory
112 Referenceable Picture Determination Unit
201 Code string analyzer
202 Prediction residual decoding unit
203 picture memory
204 motion compensation decoding unit
205 Motion vector storage unit
206 picture memory control unit
207 Working picture memory
208 Writable area determination unit
Cs computer system
FD flexible disk
FDD flexible disk drive

Claims (9)

A moving image encoding device that performs encoding of an image input with reference to an already encoded picture,
A reference-possible picture determination unit that determines a picture that can be referred to one or a plurality of encoded pictures stored in the storage area,
The referenceable picture determination unit is configured to assume the operation of the storage area in the corresponding decoding apparatus, and secure a storage area required for decoding a picture to be decoded in the decoding apparatus in the storage area. A moving image coding apparatus that determines whether or not reference is possible so that the video can be referred to. 2. The moving picture code according to claim 1, wherein the referenceable picture determination unit prohibits reference to a picture deleted after the decoding process of the target picture from a storage area of the assumed decoding device. 3. Device. The said reference possible picture determination means prohibits referring to the picture which has the oldest display order information among the pictures accumulate | stored in the storage area | region in the decoding apparatus assumed. Video encoding device. A moving image encoding device that performs encoding of an image input with reference to an already encoded picture,
A memory management command generation unit that generates a command for prohibiting use of one or more encoded pictures stored in the storage area for reference to one or more encoded pictures,
The memory management command generation means assumes that the operation of the storage area in the corresponding decoding device is assumed, and the picture deleted from the storage area after the decoding process of the picture to be decoded in the decoding device is always inaccessible. A moving image encoding apparatus for generating a command as follows. A computer-readable recording medium on which an encoded signal, which is a signal obtained by encoding a moving image, is recorded,
The coded signal is capable of performing a decoding process by sharing a storage area for decoding a picture to be decoded in a decoding apparatus with a storage area for storing decoded pictures. A recording medium characterized by including a flag indicating the following. A moving image decoding apparatus that decodes a code sequence generated by performing encoding with reference to an encoded picture,
A writable area determination unit that determines an area that can be overwritten during the decoding process of the decoding target picture for the decoded picture stored in the storage area,
A moving picture decoding apparatus, characterized in that a target picture is decoded while accumulating decoded data in an area determined to be writable by said writable area determining means. 7. The moving picture decoding apparatus according to claim 6, wherein the writable area determination unit determines, as a writable area, an area in which a picture to be deleted from a storage area after decoding processing of the current picture is stored. apparatus. The writable area determination means is not allowed to be referred to among pictures determined to be displayed in a situation where it is possible to determine whether a decoded picture in the storage area has already been displayed. The moving picture decoding apparatus according to claim 6, wherein an area in which pictures are stored is determined as a writable area. The moving picture decoding apparatus according to claim 6, wherein the writable area determination unit determines an area in which a picture having the oldest display order information in the storage area is stored as a writable area.

Images