KR100962814B1 - Video encoding method and video decoding method - Google Patents

Abstract

Encoding the image (step 100), determining whether there is unnecessary memory (step 102), encoding memory management information for opening unnecessary memory when there is unnecessary memory (step 103), The step of opening unnecessary memory (step 104), the step of determining whether the memory management information for opening the unnecessary memory by encoding of the immediately preceding image has been encoded (step 105), and if the memory management information is encoded, unnecessary again Encoding the memory management information to open the memory (step 106).

Description

Image encoding method and image decoding method {VIDEO ENCODING METHOD AND VIDEO DECODING METHOD}

The present invention relates to an image encoding method for efficiently compressing a moving image signal using correlation between screens, an image decoding method for correctly decoding the same, a program for implementing the same in software, and the like.

In recent years, in the age of multimedia, which integrates audio, images, and other pixel values, a means of delivering information such as information media, such as newspapers, magazines, television, and telephones, to humans has become a subject of multimedia. I could lift it. In general, the term "multimedia" refers not only to text, but also to a figure, voice, and in particular, images and the like. In order to make the conventional information media an object of the multimedia, it is necessary to present the information in a digital format. It becomes a condition.

By the way, when the information amount of each information media is evaluated as digital information amount, the information amount per character is 1 to 2 bytes in case of character, 64kbits per second (telephone quality) for voice, and 100Mbits per second for moving image The amount of information above (current television reception quality) is required, and it is not practical to treat the vast amount of information as it is in digital form with the information media. For example, television telephones have already been put to practical use by an Integrated Services Digital Network (ISDN) having a transmission rate of 64 kbps to 1.5 Mbps. However, it is impossible to directly send an image of a television camera to an ISDN.

Therefore, what is needed is information compression technology, for example, in the case of television telephones, a moving picture compression technology of H. 261 or H. 263 standard which is internationally standardized by ITU-T (International Telecommunication Union Telecommunication Standardization Sector) is used. It is becoming. In addition, according to the MPEG-1 standard information compression technology, it is also possible to embed image information together with audio information in a normal music CD (compact disc).

Here, MPEG (Moving Picture Experts Group) is an international standard for digital compression of moving picture signals, and MPEG-1 is a standard for compressing moving picture signals up to 1.5 Mbps, that is, about one hundredth of a television signal. . In addition, since the transmission speed for the MPEG-1 standard is mainly limited to about 1.5 Mbps, the moving picture signal is compressed to 2 to 15 Mbps in MPEG-2, which is standardized to satisfy the demand for higher image quality.

In the present situation, MPEG-4, which has a higher compression ratio, has been standardized by a working group (ISO / IEC JTC1 / SC29 / WG11) that has been standardized to MPEG-1 and MPEG-2. MPEG-4 not only enables efficient encoding at low bit rates, but also introduces powerful error-tolerance technology that can reduce subjective picture quality degradation even if transmission line errors occur. In addition, standardization of Joint Video Team (JVT) is underway as a next-generation picture coding method jointly by ISO / IEC and ITU, and at this point, what is called a joint model 2 (JM2) is the latest.

In JVT, unlike conventional moving picture coding, an arbitrary picture (picture) can be selected as a reference picture from a plurality of pictures (pictures) as a forward reference picture. Here, a picture represents a frame or a field.

Fig.1 (a) is explanatory drawing of the image coding which encodes with reference to the image selected from the many reference image stored in the memory. Fig. 1B is a block diagram showing the configuration of a memory in which an image is saved.

As shown in Fig. 1B, the memory is composed of a short time storage memory and a long time storage memory. The short-term storage memory stores some pictures decoded immediately before, and corresponds to reference pictures of so-called MPEG-1 or MPEG-2 P pictures (forward predictive coded pictures) and B pictures (bidirectional predictive coded pictures). The long time storage memory is used to store the image signal for a longer time than the short time storage memory.

Usually, the short-term storage memory is a FIFO (first-in-first-out) memory. When storing an image exceeding the upper limit of the memory in the short-term storage memory, the image of the oldest time in the short-term storage memory is removed and a new image is stored in the area. . Therefore, when it is desired to refer to a reference picture removed from the memory by a combination of FIFOs, a long-term reference is possible by moving the reference picture from the short-term storage memory to the long-term storage memory in advance and storing it in the long-term storage memory. The long-term memory is a method of specifying an area to be saved, and pictures stored in that area can be referred to unless the same area is designated and overwritten.

FIG. 1 (a) shows a prediction situation at the time of picture coding, the picture of picture number 2 refers to the picture of picture number 0, and the picture of picture number 1 refers to the picture of picture number 0 or picture number 2. . Similarly, the picture of picture number 4 refers to pictures of picture numbers 0 and 2, and the picture of picture number 6 refers to pictures of picture number 0. In the picture of picture number 5, pictures of picture numbers 0, 2, 4, and 6 can be referred to.

Incidentally, in Fig. 1A, pictures of picture numbers 0, 6, and 12 are referred to for a relatively long time, whereas pictures of picture numbers 2, 4, and 8 are only referred to from pictures after a short time. Therefore, the memory area for storing the image is divided into a short time storage memory and a long time storage memory as shown in FIG. have.

By the way, as shown in FIG. 1 (a), in order to use the memory efficiently, high memory management is required, and a configuration for controlling the memory is introduced in JVT.

The commands for controlling the memory are as follows.

1.Command for selecting a referenceable picture

2. A command to open a memory area in which a picture that is no longer needed as a reference picture of predictive encoding is stored in a short-term storage memory.

3. A command to move the contents of the short-term storage memory to the long-term storage memory

In the picture coding / decoding, since a picture having a small prediction error is selected as a reference picture from among the referenceable pictures, a signal indicating the reference picture in block units is required. By selecting a picture that can be referred to in advance, the number of candidates of the reference picture can be compressed to an appropriate value, and the number of bits of the reference picture indicating signal required in units of blocks can be saved.

In addition, when moving from the short-term storage memory to the long-term storage memory, it is unnecessary to save the same contents in both the short-term storage memory and the long-term storage memory, so that the image in the short-term storage memory is removed.

2 (a) and 2 (b) are flowcharts illustrating a conventional picture coding method and a picture decoding method.

Fig. 2A shows the operation of the picture coding apparatus when opening a memory area in which a picture that has become unnecessary as a reference picture of predictive coding is stored. In Fig. 2A, first, the image encoding apparatus encodes an input image (step 100). After encoding, an unnecessary area (image not referred to by subsequent encoding) in the memory is examined (step 101) to determine whether there is an unnecessary memory area (step 102). If it is determined that there is an unnecessary memory area (Yes in step 102), the command for opening the unnecessary memory area is encoded as memory management information (step 103), and the unnecessary memory area is opened (removing the image in the memory) (step 104) The process ends. On the other hand, when the image encoding apparatus determines that there is no unnecessary memory area (No in step 102), the processing in step 103 and step 104 is not performed, and the processing ends.

Next, the operation performed by the picture decoding apparatus when opening a memory area in which a picture that is no longer needed as a reference picture of predictive encoding is stored will be described according to the flowchart of FIG. First, the image decoding device decodes the memory management information (step 110), and decodes the image signal from the coded signal (step 111). The image decoding apparatus determines whether there is a memory open command (step 112), and if there is a memory open command (Yes in step 112), whether there is an image to be removed from the command or whether the memory open command has already been completed ( Image removal is completed) (step 113). If it is determined that the opening is completed (Yes in step 113), an error (ERROR) is set. This is because, in the JVT, a command for removing the same image again after removing the image from the memory is prohibited. Therefore, an error is caused when the opened memory is opened again. On the other hand, if it is determined that the opening is not complete (No in step 113), the image decoding device opens the memory (step 114) to end the processing. If it is determined that there is no memory open command (No in step 112), the operations in steps 113 and 114 are not performed, and the processing ends. Incidentally, the steps 110 and 111 are not in the same order and may be replaced.

3 (a) and 3 (b) are flowcharts showing another conventional image coding method and image decoding method.

Fig. 3A shows the operation performed by the picture coding apparatus when moving an image from the short time storage memory to the long time storage memory.

In Fig. 3A, first, the image encoding device encodes an input image (step 120). After encoding, it is checked whether there is an image to be moved to the storage memory for a long time (step 121), and it is determined whether there is an image to be moved (step 122). If there is an image to be moved (Yes in step 122), a command indicating how to move to the long-term storage memory is encoded as memory management information (step 123), and the image is moved to the long-term storage memory according to this command (step 124). The process ends. On the other hand, when the image encoding apparatus determines that there is no image to be moved to the long term storage memory (No in step 122), the operation of step 123 and step 124 is not performed, and the processing ends.

Next, the operation performed by the image decoding device when moving an image from the short-term storage memory to the long-term storage memory will be described according to the flowchart of FIG. 3B. First, the image decoding device decodes the memory management information (step 130), and then decodes the image signal from the coded signal (step 131). Then, the image decoding apparatus determines whether there is a command to move to the long-term storage memory in the decoded memory management information (step 132), and if it is determined (Yes in step 132), whether there is an image to be moved to this command next. Or, it is determined whether the movement is already completed (the image does not exist since the removal is completed after the movement) (step 133). In JVT, it is forbidden to send a command to move the same image back to the long-term storage memory after moving to the long-term storage memory. Therefore, an error occurs when moving the completed image to the long-term storage memory again. have. Therefore, if the image decoding apparatus determines that the movement is completed to the long time storage memory (Yes in step 133), an error (ERROR) is set. If it is determined that the movement is not completed, the image decoding apparatus moves to the long time storage memory (step 134), and the processing ends. .

On the other hand, when the image decoding apparatus determines that no command moves to the long term storage memory (No in step 132), the operations of steps 133 and 134 are not performed, and the processing ends. In addition, the steps 130 and 131 are not in the same order and may be replaced.

4 (a) and 4 (b) are flowcharts showing another conventional image encoding method and image decoding method.

First, the operation performed by the picture coding apparatus at the time of selecting a referenceable picture will be described according to the flowchart of Fig. 4A.

First, the picture coding apparatus selects a reference picture (typically a reference picture that is close in time) that is expected to have a high correlation with the skin coded picture (step 200). Next, the instruction information (a kind of memory management information) indicating the candidate of the selected reference picture is encoded (step 201). Among the candidates of the selected reference picture, the reference information is encoded with reference to the appropriate reference picture in block units (step 202). Quit. Incidentally, the steps 201 and 202 are not in the same order and may be replaced.

Next, the operation performed by the picture decoding apparatus at the time of selecting a referenceable picture will be described according to the flowchart of FIG. 4 (b).

First, the picture decoding apparatus decodes the indication information, which is a kind of memory management information (step 210), and as a result, selects a candidate of the reference picture from the memory (step 211), and selects an appropriate reference in block units from among the candidates of the selected reference picture. The image is selected and decoded with reference to it (step 212), and the processing ends.

By the way, in the conventional picture coding method and the picture decoding method, a command for removing an unnecessary picture from a memory or a command for moving a picture from a short time storage memory to a long time storage memory is encoded and output by a picture coding apparatus, and then outputted to the picture decoding apparatus. Since the number of transmissions is limited to only one picture, when the picture accompanying the command is lost due to a transmission error or the like, the arrangement of images in the memory is not correctly restored and the image cannot be decoded. .

In addition, in the encoding and decoding of a picture, when selecting a reference picture, if only a picture close in time is regarded as a reference picture candidate, scalability of decoding of the picture (example of the prediction structure in FIG. 1A) Can not decode I pictures or P pictures without decoding B pictures, or other P pictures without decoding the P pictures of picture numbers 4, 10, and 16). . That is, an image that is temporally close to an image of picture number 6 is an image of picture numbers 4 and 2, but since only an image of picture number 0 can be referred to in practice, an image of picture numbers 4 and 2 that cannot be referred to is referred to as a picture of reference image. When put in the candidate, the coding efficiency is not very good.

In addition, in the conventional image coding method, it is forbidden to transmit a command for removing an unnecessary image in the memory in conjunction with an image not stored in the memory, or for transferring a command for moving the image from the short-term storage memory to the long-term storage memory. Command transmission of memory management information is hindered. There are the following reasons for prohibiting the transmission of the command in conjunction with an image not stored in the memory. In other words, since an image that is not stored in the memory has the least importance and is highly unlikely to be decoded by scalability, the command accompanying the image that is not stored in this memory is not decoded, so that the image arrangement in the memory can be accurately restored. To avoid the missing.

SUMMARY OF THE INVENTION In order to solve the above problems, an image encoding method and an image decoding method that can be correctly restored even if some memory management information is lost due to a transmission path error, and a candidate of a reference picture to be referenced can be selected more appropriately to increase encoding efficiency. It is an object to provide a picture coding method, a picture decoding method and the like.

In order to solve the present problem, an image encoding method according to the present invention is an image encoding method for encoding by referring to a reference picture selected from a plurality of reference pictures stored in a memory, wherein the encoding target picture is referred to with reference to the selected reference picture. A picture encoding step of encoding; a management information encoding step of encoding and attaching memory management information for controlling and managing a reference picture stored in the memory to the encoded encoding target picture; and encoding the management information with the memory management information And a management information re-encoding step of re-encoding separately from the encoding in the step.

Accordingly, since the memory management information is encoded and output a plurality of times, even if a transmission path error occurs when it is transmitted to the decoding apparatus, any one of the memory management information transmitted multiple times can be considered to be transmitted and decoded. The possibility of restoration is increased.

In the management information re-encoding step, the re-coded memory management information may be accompanied by information specifying the picture to be encoded, which is accompanied by the memory management information in the management information encoding step.

As a result, when a transmission error occurs when the memory management information encoded with the picture to be encoded is first transmitted to the picture decoding apparatus, the picture to be encoded accompanying the memory management information is specified, so that a transmission error occurs at some point in time. Can be detected.

In the management information encoding step, when the memory management information is appended to an encoding target picture not stored in the memory, in the management information re-encoding step, the memory management information is stored in the memory. It may also be accompanied by the target picture.

As a result, since the memory management information accompanies the important image decoded and stored in the memory, the memory management information is reliably decoded, and the possibility of accurately reconstructing the picture increases.

The picture decoding method of the present invention is a picture decoding method for decoding with reference to a reference picture selected from a plurality of reference pictures stored in a memory, and decodes memory management information for controlling and managing the reference picture stored in the memory. If the memory area to be made unnecessary of the memory is opened in accordance with the decoded memory management information, the memory area is opened if the open memory area is not completed. If the memory area to be opened is already completed, the memory is opened. No processing is performed for the above.

Accordingly, even when memory management information indicating the removal of the picture from the memory is received a plurality of times, the picture can be correctly decoded without error processing.

A picture decoding method for decoding with reference to a reference picture selected from a plurality of reference pictures stored in a memory, wherein the memory includes a short-term storage memory having a shorter retention time of a reference picture and a retention time of a reference picture than the short-term storage memory. This long time storage memory, wherein the image decoding method decodes the memory management information for controlling and managing the reference picture stored in the memory, and stores the reference picture stored in the memory according to the decoded memory management information. In the case of moving from the short-term storage memory to the long-term storage memory, if a reference picture to be moved exists in the short-term storage memory, the reference picture is moved from the short-term storage memory to the long-term storage memory, and the reference of the moving object is stored. Picture Awards If it is not present in the short-term storage memory, the movement in the memory may not be performed.

Accordingly, even when the memory management information is received a plurality of times, the picture can be decoded correctly without error processing.

Further, as an image encoding method for encoding by referring to a reference picture selected from a plurality of reference pictures stored in the memory, a reference picture stored in the memory whose importance is equal to or higher than the encoding target picture may be encoded as a candidate of the reference picture.

As a result, the coding efficiency can be improved by selecting the candidate of the referenceable picture more appropriately.

Further, the image encoding method according to the present invention includes the steps of encoding a picture to be encoded, determining whether a reference picture which is not referenced after encoding the picture to be encoded is in a memory, and if there is a reference picture that is not referred to. And a command for opening a memory area that becomes unnecessary by being referred to, wherein the decoding device for decoding coded data encodes a command that indicates opening the unnecessary memory area after decoding the picture to be encoded; Opening the unnecessary memory area and opening the unnecessary memory area before decoding the other coded picture when encoding a separate coded picture to be encoded after the coded picture. Indicating And encoding a command.

As a result, even in the case where the first command indicating the opening of an unnecessary memory area is lost, the next command to be transmitted is executed before decoding the picture, so that the delay in executing the command can be reduced.

In addition, the image decoding method according to the present invention includes the steps of decoding memory management information for managing a memory accompanying a picture to be decoded, and processing for managing the memory before the memory management information decodes the decoded picture. A first judgment step of determining whether or not a pre-decryption command is indicated, and when the memory management information is determined as the pre-decoding command in the first decision step, whether or not a process for managing memory is completed. When it is determined that the second judging step of judging and the process of managing the memory in the second judging step are completed, the process of decoding the decoding target picture and managing the memory in the second judging step is not completed. Processing to manage the memory based on the memory management information when judged not to And decoding the decoding target picture after the operation.

As a result, even in the case where the first command indicating the opening of an unnecessary memory area is lost, the next command to be transmitted is executed before decoding the picture, so that the delay in command execution can be reduced.

In addition, the image encoding method according to the present invention comprises the steps of: encoding a picture to be encoded, a step of determining whether all of the reference pictures in a memory are not referenced after encoding the picture to be encoded, and the determining step Encoding an initialization command that is a command for deleting all of the reference pictures in the memory when it is determined that all of the reference pictures in the memory are not referenced; and deleting all the reference pictures in the memory. In the initialization step, when encoding a separate encoding target picture that is encoded after the encoding target picture, all reference pictures stored in the memory before the encoding target picture, which are deleted at the time of encoding the encoding target picture, are deleted. In-memory based on the target additional information Indicating that deleting the reference picture is characterized in that it comprises a step for coding a command in the initialization resending command.

As a result, when the initialization command is transmitted to the decoding apparatus, even if the initialization command is missing due to a transmission path error, the initialization in the memory can be normally performed according to the additional information of the initialization retransmission command.

In addition, the image decoding method according to the present invention includes the steps of decoding memory management information for managing a memory accompanying a decoding target picture, decoding the decoding target picture, and in the memory management information, a reference picture in a memory. An initialization determination step of determining whether there is an initialization command that is a command for deleting all of the data; and in the memory management information when the initialization command is not determined in the memory management information in the initialization determination step, in the memory management information, According to the additional information indicating the object to be deleted, in order to delete the reference picture stored in the memory before the separate decoding object picture, which must be initialized and deleted when the other decoding object picture decoded before the picture is decoded, Deleting the reference picture in memory An initialization retransmission determination step of determining whether or not there is an initial retransmission command; and in the initial retransmission determination step, whether all the reference pictures in the memory are deleted when the memory management information is determined to be the initial retransmission command; And an initialization completion determination step of determining, and a deletion step of deleting a picture in the memory according to the additional information when it is determined that all of the reference pictures in the memory are not deleted in the initialization completion determination step. It is done.

Thus, even when the initialization command is missed due to a transmission path error when the initialization command is transmitted to the decoding apparatus, initialization in the memory can be normally performed according to additional information of the initialization retransmission command.

Also, a recording medium on which data streams encoded in slice units with reference to a selected picture from a plurality of reference pictures stored in the memory is recorded. When a reference picture stored in the memory is removed from the memory, the object to be removed is removed. The information designating the reference picture of may be attached to at least two slices and encoded.

Accordingly, in the case where encoding is performed in units of slices, even if a transmission path error occurs when transmitted to the decoding apparatus, any one of information that designates a reference picture to be removed from the memory, which has been transmitted a plurality of times, is transmitted. Since it is considered to be decoded, the possibility of correctly reconstructing the picture in slice units increases.

Also, a recording medium in which a data stream encoded in slice units is recorded by referring to a selected picture selected from a plurality of reference pictures stored in the memory, and a target picture to be removed when the reference picture stored in the memory is removed from the memory. Information encoding designating a reference picture of the data is appended to at least two slices, and the information indicating that the slice has information designating the reference picture of the object to be removed is encoded by attaching to the slice and removed. When referring to the information specifying the reference picture of the target to be removed and referencing the information specifying the reference picture of the target to be removed, the slice which does not have information for specifying the reference picture of the target may be encoded. do.

As a result, the addition of the information can be omitted in a slice having no information for designating the picture to be removed, thereby improving the coding efficiency.

As described above, according to the picture coding method and the picture decoding method according to the present invention, the picture coding method and the picture decoding method which can be correctly restored even if some memory management information is lost due to a transmission path error, and the candidates of the reference picture reference can be seen. It is possible to realize a picture coding method and a picture decoding method which are appropriately selected to increase the coding efficiency, and have a high practical value.

In addition, the present invention can be realized as such an image encoding method and an image decoding method, and is realized as an image encoding apparatus and an image decoding apparatus using such a method, or as a recording medium on which a data stream encoded by an image encoding method is recorded. It can also be realized as a program that causes a computer to execute the steps in the picture coding method and the picture decoding method. It goes without saying that such a program can be transmitted through a recording medium such as a CD-ROM or a transmission medium such as the Internet.

In addition, this specification includes the content of the previous Japanese patent application "patent application 2002-110424", "patent application 2002-190955", "patent application 2003-49711", and US provisional application "60/377656".

According to the picture coding method and the picture decoding method according to the present invention, a picture coding method and a picture decoding method which can be correctly restored even when some memory management information is lost due to a transmission path error, and a candidate of a reference picture to be referred can be selected more appropriately. The picture coding method and the picture decoding method for improving the coding efficiency can be realized, and their practical value is high.

EMBODIMENT OF THE INVENTION Hereinafter, embodiment of this invention is described using drawing.

(Embodiment Mode 1)

First, Embodiment 1 is described.

5 is a block diagram showing the configuration of a picture coding apparatus for realizing the picture coding method according to the present embodiment.

The picture coding apparatus 100 includes a memory information control unit 101, a short-term storage memory management unit 102, a long-term storage memory management unit 103, a non-storage memory management information unit 104, and a management information encoding unit 105. And a reference image selection unit 106, a storage area designation unit 107, a reference area designation unit 108, an image memory 109, an image decoding unit 111, and an image encoding unit 110. And a variable length encoder 112, a counter 113, a counter 114, and the like.

The reference picture selection unit 106 selects a candidate of the reference picture from the external importance signal Pri and picture type information PicType, and notifies the memory information control unit 101.

The memory information control unit 101 determines, by the picture type information (PicType), whether one or both pictures (pictures) can be referenced, and instructs the reference area designation unit 108 to determine whether the picture (picture) can be referred to. From 109, the corresponding reference image is output to the image encoder 110. FIG.

The image encoder 110 encodes the input image signal Vin with reference to the reference image output from the image memory 109, and the variable length encoder 112 performs variable length encoding again to decode the image encoded stream VideoStr. Output The output of the picture coding unit 110 is further decoded by the picture decoding unit 111 to be a decoded picture, and stored in the picture memory 109 as a reference picture.

At this time, the memory location in which the decoded image can be stored in the image memory 109 is designated as follows. The memory information control unit 101 inquires of the short-term storage memory management unit 102 to specify the memory location from which the image is removed from the short-term memory, and the storage area designation unit 107 records the decoded image in the memory location. Instruction is sent to the memory 109.

The short-term storage memory management unit 102 notifies the memory information control unit unit 101 of an instruction for detecting and removing unnecessary (unreferenced) images in the short-term storage memory. The long-term storage memory management unit 103 also notifies the memory information control unit 101 of a command for moving the image in the short-term storage memory to the long-term storage memory. The command for removing the unnecessary image (opening of the memory) or the command for moving the image in the short-term storage memory to the long-term storage memory are encoded by the management information encoding unit 105 to form a memory management information stream CtlStr.

On the other hand, the counter 113 for short-term storage memory and the counter 114 for long-term storage memory, in order to prevent the memory management information from being damaged by the loss of a part of the memory management information stream CtlStr due to the transmission path error. Thus, the number of times of removing the unnecessary images or the instruction for moving the images in the short-term storage memory to the long-term storage memory is measured, so that the transfer of the command multiple times is possible as necessary.

In addition, the non-storage memory management information unit 104 manages whether or not an instruction for removing unnecessary images or a command for moving an image in the short-term storage memory to the long-term storage memory has been coded in accordance with a low importance and undecoded image. When the instruction is encoded in response to a low image, an instruction is sent to the memory information control unit 101 to accompany the image of higher importance and to encode the instruction again.

Next, the image coding method according to the first embodiment of the present invention will be described. FIG. 6 is a flowchart showing the picture coding method according to the first embodiment, and shows an operation performed by the picture coding device 100 shown in FIG. 5. 6, the same symbol is attached | subjected to the same thing as FIG. 2 (a).

The characteristic of the picture coding method shown in Fig. 6 is that memory management which opens (removes the picture) a memory area in which the picture is stored when an unnecessary picture (picture) exists as a reference picture of predictive coding in the memory. The command of information is repeatedly encoded.

By repeatedly encoding the commands of the memory management information in this way, even if the command of one of the memory management information is lost due to a transmission path error, the management information of the image stored in the memory can be restored from the command of the other memory management information. Therefore, even if there is a transmission path error, the possibility of correctly reconstructing the image increases.

In Fig. 6, first, an input image is encoded (step 100). After encoding, unnecessary regions (images not referenced by subsequent encoding) are examined in the memory (step 101), and it is determined whether there are unnecessary memory regions (step 102). If there is an unnecessary memory area (Yes in step 102), the management information encoding unit 105 encodes a command for opening an unnecessary memory area as memory management information (step 103). This unnecessary memory area is then opened (step 104). If there is no unnecessary memory area (No in step 102), the operations in steps 103 and 104 are not performed.

Next, the memory information control unit 101 accompanies the encoding of the immediately encoded image (the image before the encoding target), and determines whether or not the command for opening the unnecessary memory area is encoded as the memory management information (step 105). If the command is not coded (No in step 105), the processing ends, and if the command is coded (Yes in step 105), the management information encoding unit 105 opens the unnecessary memory area. The command is coded again as memory management information (step 106) to end the processing.

In this way, when a command (of memory management information) that opens the unnecessary memory is encoded in the encoding of the immediately preceding image, the command of the memory management information is encoded again. The memory management information encoded in addition to the encoding of the immediately preceding image and the coded memory management information recoded are respectively output from the image encoding apparatus, transmitted to the image decoding apparatus, and decoded.

In addition, in the case where a command for opening an unnecessary memory area is encoded in response to the coded signal of the image encoded immediately before in step 105, the command is coded again, but it is not accompanied by the previous image. The command may be accompanied by an image before the image, or may be repeatedly encoded as memory management information, and may be transmitted along with a plurality of images.

In addition, since the command for opening the unnecessary memory area can be transmitted a plurality of times as the memory management information, it is not necessary to necessarily transmit the coded signal of an image when encoding and transmitting the command again.

In the case of retransmitting a command of memory management information, the retransmitted command may not be in the same stream as the encoded picture, but may be transmitted as a separate stream, for example, or may be recorded in a separate area of the storage medium. do.

As described above, by transmitting a command (of memory management information) that opens an unnecessary memory area a plurality of times, even if a transmission path error occurs, any one of the commands transmitted many times is considered to be transmitted and decoded. The chances of a correct restoration increase.

(Embodiment 2)

Next, Embodiment 2 of this invention is demonstrated.

7 is a block diagram of a picture decoding apparatus for realizing the picture decoding method according to the second embodiment.

The picture decoding apparatus 200 includes a memory information control unit 201, a short-term storage memory management unit 202, a long-term storage memory management unit 203, a management information decoding unit 205, a storage area designation unit 207, And a reference area designation unit 208, an image memory 209, an image decoding unit 210, a variable length decoding unit 212, and the like.

The memory information control unit 201 determines, based on the picture type information (PicType), whether one or both of the front and rear images can be referred to the encoding target, and instructs the reference area designation unit 208. The corresponding reference picture is output from the picture memory 209 to the picture decoding unit 210.

The variable length decoding unit 212 decodes an encoded stream VideoStr, and the image decoding unit 210 decodes it again, outputs it as a decoded picture signal Vout, and stores it as a reference picture in the picture memory 209. .

At this time, the memory location in which the decoded image can be stored in the image memory 209 is specified as follows. The image memory 209 causes the memory information control unit 201 to inquire the short-term storage memory management unit 202 to specify the memory location from which the image is removed, and the storage area designation unit 207 records the decoded image at the memory location. Instruct)

The management information decoding unit 205 decodes the memory management information stream CtlStr and, via the memory information control unit 201, sends information of unnecessary (unreferenced) images in the short-term storage memory to the short-term storage memory management unit 202. The long-term storage memory management unit 203 notifies a command for moving the image in the short-term storage memory to the long-term storage memory.

Next, the image decoding method in Embodiment 2 of this invention is demonstrated. FIG. 8 is a flowchart showing an image decoding method according to the second embodiment, and illustrates an operation performed by the image decoding apparatus 200 shown in FIG. 7. In Fig. 8, the same operations as those in Fig. 2B are given the same symbols.

When the picture coding apparatus transmits a command for opening an unnecessary memory area many times, the picture decoding apparatus receives a command for opening a region of the same picture in the memory many times unless the command is lost due to a transmission path error. do. Therefore, even when the image decoding apparatus receives a command for reopening a memory area that has already been opened, the image decoding method must be implemented to determine that the received image is correctly received without processing as an error. In this embodiment, such a picture decoding method is realized.

In Fig. 8, first, the management information decoding unit 205 decodes the memory management information (step 110). Next, the image signal is decoded from the coded signal (step 111). The memory information control unit 201 then determines whether there is a command for opening the memory in the decoded memory management information (step 112). If there is a command to open the memory (Yes in step 112), it is determined whether there is an image to be removed by the command or whether it is already open (deleted) (step 113). The process ends without processing, otherwise, the memory is opened (step 114) to end the process. On the other hand, when there is no memory open command (No in step 112), the operation of step 113 and step 114 is not performed, and the processing ends. Note that the steps 110 and 111 are not in the same order and may be replaced.

By the above operation, even if the command which opens the area | region of the same image in a memory is encoded and transmitted many times by the image coding method of Embodiment 1, and the image decoding apparatus 200 receives this many times, it is an error. Since the processing is not carried out as an example, an image decoding method that can be decoded correctly can be realized.

In addition, the command for opening the unnecessary memory area only needs to be transmitted a plurality of times as the memory management information, and it is not necessary to necessarily transmit the coded signal of an image when encoding and transmitting this command again.

When retransmitting the command of the memory management information, the retransmitted command may not be in the same stream as the coded picture, but may be transmitted as a separate stream, for example, or may be recorded in a separate area of the storage medium. do.

(Embodiment 3)

Next, the image coding method according to the third embodiment will be described. FIG. 9 is a flowchart showing the picture coding method according to the third embodiment, and shows the operation performed by the picture coding device 100. FIG. In Fig. 9, the same operations as Fig. 3A are given the same symbols.

A feature of the present embodiment is that when there is an image to be moved from the short-term storage memory to the long-term storage memory in the memory, the command of the memory management information for moving the image is repeatedly encoded. By repeatedly coding the command of the memory management information, even if the command of one of the memory management information is lost due to a transmission path error, the management information of the image stored in the memory can be restored from the command of the other memory management information. Even if there is an error, the possibility of accurately reconstructing the image increases.

In Fig. 9, first, an input image is encoded (step 120). It is checked whether there is an image to be moved to the storage memory for a long time after the encoding (step 121). Then, the memory information control unit 101 determines whether there is an image moving to the long term storage memory (step 122). If there is an image to be moved (Yes in step 122), the management information encoding unit 105 encodes a command indicating how to move to the long term storage memory as the memory management information (step 123). Then, the image is moved to the long-term storage memory as the command (step 124).

Next, the memory information control unit 101 determines whether or not the command that moves to the long-term storage memory in association with the coded signal of the immediately coded image (the image before the encoding target) is encoded as the memory management information (step 125). If it is not encoded (No in step 125), the process ends, and if it is encoded (Yes in step 125), the management information encoding unit 105 encodes the command to move to this long-term storage memory again as memory management information. (Step 126), the processing ends.

In this way, when a command (of memory management information) moving to the long-term storage memory is encoded by encoding the immediately preceding image, the command of the memory management information is encoded again. The memory management information and the coded memory management information which have been encoded in addition to the encoding of the immediately preceding image are respectively output from the image encoding apparatus, transmitted to the image decoding apparatus, and decoded.

In addition, the command is coded again when a command moving to the storage memory for a long time in response to the coded signal of the image coded immediately before is coded in step 125. May be repeated, and the command may be repeatedly encoded as memory management information and attached to a plurality of images.

In addition, since a command moving to the long-term storage memory can be transmitted a plurality of times as memory management information, it is not always necessary to accompany the coded signal of an image when transmitting the command again.

In the case of retransmitting the command of the memory management information, the retransmitted command may not be in the same stream as the encoded picture but may be transmitted as a separate stream, for example, or may be recorded in a separate area of the storage medium. do.

By encoding and transmitting a command for moving an image to the long-term storage memory as described above many times, even if a transmission path error occurs, any one of the commands transmitted many times is considered to be transmitted and decoded, so that the image is correctly restored. The likelihood of doing so increases.

(Embodiment 4)

Next, the image decoding method of the fourth embodiment will be described.

When the picture coding apparatus transmits a command for moving to the long-term storage memory a plurality of times, unless the command is lost due to a transmission path error, the picture decoding apparatus executes a command for moving the area of the same image in the short-term storage memory to the long-term storage memory. You will receive a number of times. Therefore, even when the image decoding apparatus receives a command for moving the image which has already been moved again, the image decoding method must be implemented to determine that the image decoding apparatus is correctly received without processing as an error. The feature of the picture decoding method in the present embodiment is such a picture decoding method.

FIG. 10 is a flowchart showing the image decoding method according to the fourth embodiment, and shows the operation of the image decoding apparatus 200 shown in FIG. 7. In Fig. 10, the same operations as Fig. 3B are given the same symbols.

10, first, the management information decoding unit 205 decodes the memory management information (step 130). Then, the image signal is decoded from the coded signal (step 131).

Then, the memory information control unit 201 determines whether or not the decoded memory management information contains a command for moving the image to the long-term storage memory (step 132). If there is a command to move to the long-term storage memory (Yes in step 132), it is determined whether there is an image to be moved to this command or whether the movement has already been completed (the image does not exist because it is removed after moving) (step 133). If the move to the storage memory is complete (Yes in step 133), the processing ends without any processing. Otherwise, the processing moves to the storage memory for a long time (step 134) to end the processing.

On the other hand, if there is no command to move to the storage memory for a long time (No in step 132), the operation of step 133 and step 134 is terminated without ending the processing. The steps 130 and 131 are not in the same order, and may be replaced.

By the above operation, even if the command for moving an image to the long-term storage memory is encoded and transmitted many times, the image decoding method which can be decoded correctly can be realized by the image coding method of the third embodiment.

In addition, since the command moving to the long-term storage memory needs to be transmitted a plurality of times as the memory management information, it is not always necessary to accompany the coded signal of the image when the command is encoded and transmitted again.

In the case of retransmitting the command of the memory management information, the retransmitted command may not be in the same stream as the encoded picture, but may be transmitted as a separate stream, for example, or may be recorded in a separate area of the storage medium. .

(Embodiment 5)

Next, the image coding method in the present embodiment will be described. FIG. 11 is a flowchart showing the picture coding method according to the fifth embodiment, and shows the operation of the picture coding device 100 shown in FIG. In Fig. 11, the same operations as in Fig. 6 are given the same symbols.

The feature of this embodiment shown in Fig. 11 is that if an unnecessary image exists in the memory, the command of the memory management information for removing the image is repeatedly encoded, and at least once attached to an important screen stored in the memory. To transmit. Even when the command of the memory management information is encoded repeatedly, when the command of the memory management information is transmitted in conjunction with an image of non-high importance, the command of the memory management information is executed when all the images of non-high importance are decoded. It cannot be acquired.

For example, in Fig. 1 (a), since the picture of picture number 4 becomes unnecessary after the picture of picture number 5 is encoded, the memory area in which the picture of picture number 4 accompanies the picture of picture number 5 is located. Open commands can be encoded.

However, in addition to encoding and attaching a command for opening a memory area in which a picture of picture number 4 is located, the picture of picture number 7 is all the least important (not decoded). If not, the command is coded in addition to the B picture having a low image quality deterioration. Since these B pictures are not decoded, the command for opening the memory area in which the picture of picture number 4 is located is not decoded, and the management information in the memory cannot be correctly reproduced. Therefore, it is necessary to encode a command that opens the image region at least once in a high degree of importance and is necessarily decoded and accompanying the image stored in the memory.

In Fig. 11, first, an input image is encoded (step 100). After encoding, unnecessary regions (images not referred to by subsequent encoding) in the memory are examined (step 101) to determine whether there are unnecessary memory regions (step 102). If there is an unnecessary memory area (Yes in step 102), the management information encoding unit 105 encodes a command for opening an unnecessary memory area as memory management information (step 103). This unnecessary memory area is then opened (step 104). If there is no unnecessary memory area (No in step 102), the operations in steps 103 and 104 are not performed.

Next, the memory information control unit 101 determines whether a command that opens an unnecessary memory area, which has been encoded in the past, is encoded by being accompanied by an important image (decoded and stored in the memory) (step 140). If it is attached and encoded (Yes in step 140), the process ends, and if it is attached to an important image and not encoded (No in step 140), the management information encoding unit 105 issues a command to open this unnecessary memory area again. The process is terminated by encoding as the memory management information (step 141).

As a result, a command for opening an unnecessary memory area is encoded with an important image.

As described above, since the command is appended to an important image decoded and stored in the memory, the command is decoded, so that the possibility of correctly reconstructing the image when a transmission path error occurs is increased.

In addition, the command for opening the unnecessary memory area can be transmitted as the memory management information a plurality of times. When the command is encoded and transmitted again, the command is not necessarily accompanied by the coded signal of the image.

When retransmitting the command of the memory management information, the retransmitted command may not be in the same stream as the coded picture, but may be transmitted as a separate stream, for example, or may be recorded in a separate area of the storage medium. do.

(Embodiment 6)

Next, the image coding method of the present embodiment will be described. FIG. 12 is a flowchart showing the picture coding method according to the sixth embodiment. FIG. 12 shows the operation of the picture coding apparatus 100 shown in FIG. In Fig. 12, the same operations as Fig. 9 are given the same symbols.

The feature of this embodiment shown in FIG. 12 is that the command of the memory management information for moving the image to the long-term storage memory is repeatedly encoded, and transmitted at least once on an important screen (decoded and stored in the memory). will be. Even when the command of the memory management information for moving the image to the long-term storage memory is repeatedly encoded, when the command of the memory management information is transmitted in conjunction with an image of low importance, all the non-high importance images are not decoded. In this case, the command of the memory management information cannot be obtained.

In Fig. 12, first, an input image is encoded (step 120). After encoding, it is checked whether there is an image to be moved to the storage memory for a long time (step 121), and it is determined whether there is an image to be moved (step 122).

If there is an image to be moved (Yes in step 122), a command indicating how the management information encoding unit 105 moves to the long-term storage memory is encoded as the memory management information (step 123), and the image is stored for a long time according to this command. Go to step 124.

Next, the memory information control unit 101 determines whether the command moving to the long-term storage memory encoded in the past is accompanied by the important image (decoded and stored in the memory) and encoded (step 150). Is completed (Yes in step 150), the processing ends, and if it is not attached to the important image (No in step 150), the command for the management information encoding unit 105 to move to the long-term storage memory again is returned to the memory management information. Is encoded (step 151), and the process ends.

As a result, a command for moving an image in the long-term storage memory is encoded along with the important image.

As described above, since the command accompanies the important image to be decoded and stored in the memory, the command is decoded, so that the possibility of correctly reconstructing the image when a transmission path error occurs is increased.

In addition, since the command moving to the long-term storage memory needs to be transmitted a plurality of times as the memory management information, it is not always necessary to be accompanied by the coded signal of the image when the command is encoded and transmitted again.

In the case of retransmitting the command of the memory management information, the retransmitted command may not be in the same stream as the encoded picture, but may be transmitted as a separate stream, for example, or may be recorded in a separate area of the storage medium. do.

(Embodiment 7)

The image coding method in the seventh embodiment will be described.

A feature of this embodiment is an image encoding method for encoding with reference to a reference image selected according to the importance of the image.

Fig. 13 is a flowchart showing the picture coding method according to the seventh embodiment of the present invention. FIG. 13 shows the operation performed by the picture coding apparatus 100 shown in FIG.

In Fig. 13, first, the importance of each image to be encoded is set (step 160). For example, the importance of the I picture and the P picture is high, and the importance of the B picture is low. In addition, even with the same P picture, the importance of the P picture referenced from many images is high, and the importance of the P picture not very referenced is low.

Next, an image of equal importance or higher than the encoding target image is selected from the reference image in the memory to be a candidate of the reference image (step 161). For example, a B picture may refer to an I picture and a P picture, but the P picture excludes P pictures of low importance from candidates of the reference picture.

Next, the indication information (a kind of memory management information) indicating the candidate of the selected reference picture is encoded (step 162), and among the candidates of the selected reference picture is referred to and encoded with reference to an appropriate reference picture in block units (step 163). In addition, step 162 and step 163 are not the same in order, and may be replaced.

In this way, the candidate of the reference image is not including an image having a lower importance than the importance of the encoding target image.

As described above, since the candidate of the reference picture does not include an image having a lower importance than that of the encoding target picture, an unreferenced picture can be excluded from the candidate of the reference picture when a stream capable of achieving scalability is generated. The coding efficiency is improved.

Here, the picture encoding method performed according to the importance of the picture set as described above will be described in detail with reference to FIG.

Fig. 14A shows a number (picture (frame) number) assigned to each frame, a number when each frame is stored in a memory (preservation picture (frame) number), and a number indicating the order in which each frame is transmitted. It is explanatory drawing which shows the relationship of (transmission order).

In Fig. 14A, the I picture of picture number 0 is stored in the memory because it does not refer to other pictures, and the stored picture number is zero. Next, since the P picture of picture No. 2 referring to the I picture of picture number 0 is stored in the memory, the stored picture number for the P picture of picture number 2 is set to one. Then, since the B picture of picture number 1 which refers to the I picture of picture number 0 and the P picture of picture number 2 is stored in the memory, the save picture number of the B picture of picture number 1 becomes two. The order in which the pictures are transmitted is in the order in which they are stored in the memory. In the same order, the relationship between the picture number, the stored picture number, and the transmission order is determined.

Next, an example of the relationship between the decoded (decoded) picture number, the picture number stored in the memory, and the picture number to be deleted will be described with reference to Fig. 14B.

Fig. 14B is a relation diagram showing the relationship between the decoded picture number (frame number), the stored picture number (frame number) and the deleted picture number (frame number). In this case, the maximum number of pictures that can be stored in the memory is five. Pictures are stored in the memory in the order of transfer.

In addition, for example, when a P picture having a picture number of 4 is decoded, since a stored picture number of a P picture having a picture number of 4 is 3, a picture having 0, 1, or 2 stored picture numbers is stored on the memory. do. When the B picture whose picture number to be decoded is 3 is decoded, as shown in Fig. 14B, pictures having picture numbers of 4, 1, 2, and 0 are stored. Here, the picture B of picture number 1 is not referred to from any picture after decoding the picture of picture number 3, as shown in Fig. 14A, so that the picture having picture number 3 is decoded. Is deleted from.

Similarly, when a B picture having a decoded picture number of 5 is decoded, as shown in Fig. 14B, pictures having picture numbers of 6, 3, 4, 2, and 0 are stored. Here, since the picture B of picture number 3 is not referred to from any picture after decoding the picture of picture number 5, the B picture with picture number 5 is deleted at the time when it is decoded.

In addition, when a P picture whose picture number to be decoded is 8 is decoded, as shown in Fig. 14B, pictures having picture numbers 5, 6, 4, 2, and 0 are stored. Since only up to five frames can be stored in the memory, in order to refer to the P picture with picture number 8 later, the P picture with picture number 8 is deleted by deleting any one of picture numbers 5, 6, 4, 2, and 0. You must secure memory to remember. Therefore, as the selection criterion for erasing a frame in Fig. 14B, in the decoding of the P picture, that is, the decoding of the even-numbered picture number, the picture that is the oldest in time, that is, the I picture whose picture number is 0 in this case, Is deleted at the time when the P picture of picture number 8 was decoded.

Similarly, when a B picture having a decoded picture number of 7 is decoded, as shown in Fig. 14B, pictures having picture numbers of 8, 5, 6, 4, and 2 are stored. Here, since the B picture of picture number 5 is not referenced from any picture after decoding the B picture of picture number 7, it is deleted at the time when the B picture with picture number 7 is decoded.

When a P picture having a picture number of 10 decoded is decoded, as shown in Fig. 14B, pictures having picture numbers 7, 8, 6, 4, and 2 are stored. Since only 5 frames can be stored in the memory at a time, in order to refer to a P picture having picture number 10 later, the picture having picture number 10 is deleted by deleting any one of picture numbers 7, 8, 6, 4, and 2. You must secure memory to remember. Therefore, as the selection criteria of the frame to be deleted in Fig. 14B, in the decoding of the P picture, that is, the decoding of the picture which is the picture number of the even number, the picture having the oldest picture number is 2 as the picture number 10. The P picture is deleted at the time of decoding.

When the picture is deleted in this manner, a command of the memory management information for deleting the picture is encoded and attached to the coded signal of the picture to be decoded and transmitted.

In the example shown in Fig. 14B, an example is described in which an unnecessary picture (picture) exists in the memory and a command of memory management information for removing the picture is sent once. In this way, if the command of the memory management information to be removed is sent only once, there is a possibility that the command of the memory management information sent accompanying the B picture cannot be executed. Because the B picture is unlikely to be used as an image referred to in the encoding and decoding of the P picture, it is highly likely that the data of the B picture is preferentially discarded when sufficient storage capacity or transmission capacity cannot be secured. As a result, there is a possibility that the command of the memory management information sent to the B picture cannot be executed.

To solve this problem, an example in which a command of memory management information for removing an image is repeatedly encoded and transmitted will be described. Hereinafter, FIG. 14 (c) will be described in detail.

Fig. 14C is a relation diagram showing another relationship between the decoded picture number (frame number), the stored picture number (frame number) and the deleted picture number (frame number). In Fig. 14C, the command for deleting the picture of the picture number to be deleted accompanies the coded signal of the picture of the picture number to be decoded.

As shown in Fig. 14C, when a B picture having a picture number of 3 is decoded, a picture having picture numbers of 4, 1, 2, and 0 is stored. Here, the B picture with picture number 1 is not referenced from any picture after decoding the picture with picture number 3, as shown in Fig. 14A. Therefore, when the picture having picture number 3 is decoded, the B picture of picture number 1 is deleted, and a command of the memory management information for deleting is attached to the picture having picture number 3.

However, since the picture with picture number 3 is a B picture, it is of lower importance as described above in terms of image reproduction compared to an I picture or a P picture, and data is easily destroyed at the time of transmission, thus accompanying the B picture of picture number 3 There is a possibility that the command of the sent memory management information cannot be executed (when the frame is saved as shown in FIG. 25).

Therefore, the memory management information indicating that the picture of picture number 1, which is attached to picture number 3, is deleted from the P picture of picture number 6, which is more important in terms of image reproduction than the picture B of picture number 3, which is decoded next. Break the command (see Figure 14 (c)).

Similarly, a P picture of picture number 8 is accompanied by a command of memory management information (indicating that the picture of picture number 3 is deleted) attached to a B picture of picture number 5, and a picture of picture number 7 is assigned to a P picture of picture number 10. The command of the memory management information (shown to delete the picture of picture number 5) accompanying the B picture is appended. Since the picture of picture number 8 is a P picture, as shown in Fig. 14C, the B picture of picture number 7 does not accompany the command of memory management information attached to the picture of picture number 8, You can also do it.

As shown in Fig. 14 (c), the command of the memory management information that is the same as the command of the memory management information first attached to the B picture is a picture of higher importance in terms of image reproduction than the B picture. The picture to be stored or transmitted after the B picture that accompanied the command of the memory management information is repeatedly appended. Thus, even if the B picture in which the command of the memory management information is attached first is dropped, the command of the memory management information can be executed normally.

As described with reference to Fig. 14 (c), even when a command of the memory management information is attached to the B picture, the set importance is used when the command of the repetitive memory management information is attached to the P picture again. In addition, setting of importance is not limited to what was shown to this embodiment.

In the present embodiment, the importance of each image is determined according to the importance of each image, and the importance in each image is encoded along with each image as in the memory management information shown in the above embodiment. no. Therefore, the decoding process of the encoded data in this embodiment is not different from the conventional method.

(Embodiment 8)

Next, Embodiment 8 will be described.

A feature of the present embodiment is that the entire picture (picture) in the memory is deleted, and a command (of memory management information) for performing initialization of the memory area is encoded and transmitted many times.

The memory management information shown in each of the above embodiments is given as code information as shown in FIG.

Fig. 15 is a correspondence diagram showing a command of the memory management information, which shows a code number (Code), the content of the command (command), and its additional information (additional information).

For example, a command for opening an unnecessary memory area of the short-term storage memory (short-term storage memory opening) is given as code information Code1, and a picture number (frame number) to open is added as additional information.

In addition, code information is given as header information of each frame shown in FIG.

17 is a schematic diagram illustrating a relationship between header information and frame data in the coded signal of each picture. In FIG. 17, each coded signal represents a coded signal of frames Frm12, Frm11, and Frm14 described later. Each coded signal includes a frame header having header information and frame data relating to encoding of an image. For example, the coded signal of the frame Frm12 includes a frame header Frm12Hdr and frame data consisting of data MB12a, MB12b, MB12c, MB12d, and the like.

The detail of this coded signal is shown in the schematic diagram of FIG.

18 is a schematic diagram showing a command of memory management information in header information of an encoded signal.

As shown in FIG. 18, the coded signal of frame FrmA is provided with the frame header FrmAHdr which has header information, and frame data which consists of each data MBa, MBb, MBc, MBd. Then, the additional information AddA of the code information CodeA after the code information CodeA of the command is followed by the code information CodeB of the command to be executed after the command of the code information CodeA and the code information. The additional information AddB of (CodeB) is added to the frame header FrmAHdr. If there is no additional information, only code information is added, such as code information CodeC.

Next, the procedure of command execution is shown in FIG.

16 is a flowchart showing the command execution procedure.

In Fig. 16, first, a command is acquired (step C0), and it is determined whether or not the acquisition of the command is completed (step C1). When the command is not acquired and the command is acquired (No in step C1), the acquired command is executed (step C2), and the operation returns to step C0 and the operation is repeated. On the other hand, if the command acquisition ends and the command is not acquired (Yes in step C1), the command execution processing ends. This procedure is performed for each frame. In addition, even when command information is sent in units of slices composed of a plurality of large blocks, the commands are executed in the same order as described above.

By the way, in Embodiment 1, the command of the memory management information (opening the memory) for removing unnecessary images has been described. In Embodiment 1, by repeatedly encoding a command of memory management information for removing an unnecessary image, even if a command of one memory management information is lost due to a transmission path error, it is stored in the memory from a command of the other memory management information. It has been shown that the management information of the existing image can be restored, and the possibility of correctly reconstructing the image increases.

Here, among the code information shown in FIG. 15, the initialization command Code5 for removing all the information in the memory will be examined.

When the initialization command Code5 is sent only once, if this initialization command Code5 is lost due to a transmission path error, it affects processing such as memory management performed after the initial initialization. Therefore, as in the first embodiment, the case where the initialization command Code5 is repeatedly encoded and transmitted will be described with reference to FIG. 19.

Fig. 19 shows a number (picture (frame) number) assigned to each frame, a number (storage picture (frame) number) when each frame is stored in memory, and a number (transmission) showing the order in which each frame is transmitted. It is explanatory drawing which shows the relationship of order).

Hereinafter, FIG. 19 is demonstrated concretely. First, the I picture of picture number 0 is stored in the memory because it does not refer to other pictures, and the stored picture number is zero. Next, since the P picture of picture number 2 referring to the I picture of picture number 0 is stored in the memory, the stored picture number for the P picture of picture number 2 is set to 1. Then, since the B picture of picture number 1 which refers to the I picture of picture number 0 and the P picture of picture number 2 is stored in the memory, the save picture number of the B picture of picture number 1 becomes 2. The order in which the pictures are transmitted is in the order stored in the memory. In the same order, the relationship between the picture number, the stored picture number, and the transmission order is determined.

When the I picture of picture number 12 shown in FIG. 19 is encoded, an initialization command Code5 shown in FIG. 15 is sent. Since the stored picture number of the I picture of picture number 12 is 11, this initialization command (Code5) allows all pictures having the stored picture number 10 or less to be removed from memory.

Here, a method of encoding the initialization command Code5 will be described with reference to FIG. 20. FIG. 20 is a flowchart showing a method of encoding the initialization command Code5, and shows an operation performed by the image coding apparatus 100 shown in FIG.

First, the input image is encoded (step A0). After encoding (irrespective checking) whether all referenceable pictures in the memory are unnecessary (no image is referenced in subsequent encoding) (step A1), the pictures stored in the memory are initialized without being referenced later. It is determined whether one is good (step A2).

If it is better to initialize (Yes in step A2), the initialization command Code5 for initializing the memory area is encoded as memory management information (step A3), and initialization is performed (step A4), and the processing ends. On the other hand, when there is no need for initialization (No in step A2), the operation of step A3 and step A4 is not performed, and the processing ends.

Next, a method of decoding the encoded initialization command Code5 will be described with reference to FIG.

FIG. 21 is a flowchart showing a method of decoding an encoded initialization command Code5, and illustrates an operation performed by the image decoding device 200 shown in FIG.

First, the memory management information is decoded (step A10), and the image signal is decoded from the coded signal (step A11). Next, it is determined whether the decoded memory management information has an initialization command Code5 (step A12). If there is an initialization command Code5 (Yes in step A12), all pictures stored in the memory are removed and initialized. (Step A13), processing ends. At this time, however, the decoded image (in step A11) is not removed.

On the other hand, if there is no initialization command Code5 in the memory management information (No in step A12), the processing ends.

Hereinafter, a method of initializing a memory will be described in detail with reference to FIG. 19. An initialization command Code5 identical to the initialization command Code5 assigned to the picture I of picture number 12 is assigned to the B picture of picture number 11 shown in FIG.

In FIG. 17, the initialization command Code5 is provided to the frame header Frm12Hdr of the frame Frm12 (picture number 12) and the frame header Frm11Hdr of the frame Frm11 (picture number 11). Since the initialization command Code5 does not have additional information as shown in FIG. 15, all pictures stored in the memory at the time of decoding are removed.

Therefore, the initialization command Code5 assigned to the I picture of picture number 12 (preserved picture number 11) is lost due to a transmission error, and the initialization command Code5 assigned to the B picture of picture number 11 (preserved picture number 12) is executed. In this case, all pictures stored in the memory as pictures decoded before the stored picture number 11 are removed. That is, up to the I picture of picture number 12 (preserved picture number 11) that should not be originally removed.

In this way, when the same initialization command (Code5) as the initialization command (Code5) assigned to the picture B of the picture number 11 is assigned to the I picture of the picture number 12, one picture (I picture of picture number 12) is lost. . On the other hand, to the P picture of picture number 14 (preserved picture number 13), the same initialization command (Code5) as that assigned to I picture of picture number 12 (preserved picture number 11) is assigned to code picture 12, and If the initialization command Code5 assigned to the I picture is lost due to a transmission error, and the initialization command Code5 assigned to the P picture of picture number 14 is executed, two pictures (B picture of picture number 11 and picture number 12) are executed. I picture) is eliminated.

The same problem occurs even when the initialization command Code5 is repeatedly encoded and the initialization command Code5 sent first and the initialization command Code5 sent continuously are also executed without a transmission path error. This is because it is initialized by the initializing command Code5 sent first, and again by the initializing command Code5 sent continuously.

A method for solving the problem in the initialization of such a memory will be described.

Fig. 22 shows a command of memory management information used to solve a problem in memory initialization.

The difference from FIG. 15 is that in FIG. 22, a newly initialized retransmission command Code6 is added. In addition, this initialization retransmission command Code6 has an initialization picture (frame) number (number of frames accompanied by an initialization command Code5 for initializing a memory area) as additional information.

Hereinafter, the flow of image coding processing using this initialization retransmission command Code6 will be described with reference to FIG. 23.

FIG. 23 is a flowchart showing a picture coding method using the initialization retransmission command Code6, and shows the operation performed by the picture coding device 100 shown in FIG. In FIG. 23, the same reference numerals are assigned to the same operations as FIG. 20.

First, the input image is encoded (step A0). After the encoding, whether or not all the pictures in the memory are unnecessary (no image is referred to by subsequent encoding) is checked (initializing irradiation) (step A1). The memory information control unit 101 determines whether initialization is necessary (step A2), and if initialization is required (Yes in step A2), the management information coding unit 105 manages an initialization command Code5 for initializing the memory area. It encodes as information (step A3) and performs initialization (step A4). If no initialization is necessary (No in step A2), the operations of step A3 and step A4 are not performed.

Next, it is determined whether or not the memory information control unit 101 encodes, as memory management information, the initialization command Code5 for accommodating the coded signal of the immediately coded image (the image before the encoding target) to initialize the memory area. (Step A30) If encoding is performed (Yes in step A30), the management information encoding unit 105 encodes the initialization retransmission command Code6 for initializing the memory area as the memory management information (step A31) to perform the processing. Quit.

If the initialization command Code5 for initializing the memory area accompanying the coded signal of the immediately coded image (the image before the encoding target) is not encoded as the memory management information (No in step A30), the processing ends. .

In addition, in the method shown in FIG. 23, when the initialization command Code5 which initializes a memory area accompanying the coded signal of the image encoded just before is coded, it is supposed to code the initialization retransmission command Code6 again. When the initialization command Code5 for initializing the memory area is encoded in response to the encoding of the image encoded before a few images, not just before, the initialization retransmission command Code6 may be coded again, or in addition to a plurality of images. The initialization retransmission command Code6 for initializing the memory area may be repeatedly encoded as the memory management information.

Specifically, as shown in FIG. 19, when the initialization command Code5 is encoded in accordance with the encoding of the I picture of picture number 12, the initialization retransmission command Code6 is encoded in addition to the encoding of the B picture of picture number 11. Alternatively, the initialization retransmission command Code6 may be encoded in addition to encoding of the P picture of picture number 14.

In the former case, as shown in FIG. 17, the initialization command Code5 is assigned to the frame header Frm12Hdr of the frame Frm12, and the initialization retransmission command Code6 is assigned to the frame header Frm11Hdr of the frame Frm11. In the latter case, the initialization command Code5 is assigned to the frame header Frm12Hdr of the frame Frm12, and the initialization retransmission command Code6 is provided to the frame header Frm14Hdr of the frame Frm14.

In addition, the initialization retransmission command Code6 may be coded in conjunction with the coding of the B picture of picture number 11, and the initialization retransmission command Code6 may be encoded in addition to the coding of the P picture of picture number 14. In this case, as shown in FIG. 17, the initialization command Code5 is attached to the frame header Frm12Hdr of the frame Frm12, the frame header Frm11Hdr of the frame Frm11, and the frame header Frm14Hdr of the frame Frm14. ) Is assigned an initialization retransmission command Code6.

Next, the process at the time of decoding the data which the said initialization retransmission command Code6 encoded is demonstrated using FIG. FIG. 24 is a flowchart showing a method of decoding the encoded initialization retransmission command Code6, and shows the operation of the picture decoding apparatus 200 shown in FIG. In Fig. 24, the same reference numerals are assigned to the same operations as in Fig. 21.

First, the management information decoding unit 205 decodes the memory management information (step A10). Then, the image signal is decoded from the coded signal (step A11).

It is determined whether the decoded memory management information has an initialization command (Code5) (step A12), and if there is an initialization command (Code5) (Yes in step A12), all pictures in the memory are removed and initialized (step A13). If there is no command Code5 (No in step A12), no initialization is performed.

Next, the memory information control unit 101 determines whether the initialization retransmission command Code6 is present in the memory management information (step A40). If there is no initialization retransmission command Code6 (No in step A40), the process ends, and if there is an initialization retransmission command Code6 (Yes in step A40), it is checked whether or not initialization is completed (step A41). If the initialization is completed (Yes in step A41), the processing ends, and if it is not initialized (No in step A41), the initialization frame (initialization to initialize the memory area) is based on additional information of the initialization resend command (Code6). The preservation frame (frame stored in the reference picture memory at the time of encoding the initialization frame) before the frame where the command Code5 is attached is deleted, and the processing is terminated by setting the preservation memory size to 0 (step A42). do. If the long-term storage frame is not used, it is not necessary to set the long-term storage memory size to zero.

Therefore, when the initialization command (Code5) and the initialization retransmission command (Code6) are attached to the picture of picture number 12 shown in FIG. 19 and encoded, the initialization command (Code5) is not lost due to transmission path error. When the initialization command Code5 is lost due to a transmission error by the initialization command Code5, all pictures stored in the memory are deleted by the initialization retransmission command Code6 with a picture having a stored picture number of 10 or less. Will be.

In this way, when the initialization command Code5 is repeatedly encoded and transmitted, since the second time or later, the initialization retransmission command Code6 to which the initialization picture number as additional information is added is encoded and transmitted. The storage frame (the frame stored in the reference picture memory at the time of encoding the initialization frame initially accompanied by the initialization command Code5) is deleted. For this reason, the above-mentioned problem that a required image (picture) is missing can be solved.

In addition, even when a storage picture number different from that in FIG. 19 is assigned as in FIG. 25, the initialization retransmission command Code6 described above is valid.

Hereinafter, this will be described in detail.

Fig. 25 is a number (picture (frame) number) assigned to each frame, a number when each frame is stored in a memory (preservation picture (frame) number), and a number (transmission) showing the order in which each frame is transmitted. It is explanatory drawing which shows the other relationship in order).

The method of attaching these numbers is demonstrated. First, the I picture of picture number 0 is stored in the memory because it does not refer to other pictures, and the stored picture number is zero. Next, since the P picture of picture number 2 referring to the I picture of picture number 0 is stored in the memory, the stored picture number for the P picture of picture number 2 is set to one. The picture I of picture number 0 and the picture B of picture number 1, which refer to the P picture of picture number 2, are stored in the memory. Since this B picture is not referred to from other pictures, the stored picture number is stored immediately before. It becomes 1 similarly to the stored picture number of the P picture of the completed picture number 2. The order in which the pictures are transmitted is in the order stored in the memory. In the same order, the relationship between the picture number, the stored picture number, and the transmission order is determined.

As shown in FIG. 25, when the I picture of picture number 12 is encoded, the initialization command Code5 shown in FIG. 15 is sent along with it. Since the stored picture number of the I picture of picture number 12 is 6, this initialization command (Code5) enables the removal of all pictures having a stored picture number of 5 or less in the memory.

Here, in the case where the initialization command Code5 is repeatedly encoded, a case where the same initialization command Code5 as the initialization command Code5 assigned to the I picture of picture number 12 is given to the P picture of picture number 14 is described. do.

Since the initialization command Code5 does not have additional information as shown in FIG. 15, all pictures stored in the reference memory at the time of decoding are removed. Therefore, the initialization command Code5 assigned to the I picture of picture number 12 (preserved picture number 6) is lost due to a transmission error, and the initialization command Code5 assigned to the P picture of picture number 14 (preserved picture number 7) is executed. In this case, all the pictures stored in the memory are removed with pictures having the stored picture number 6 or less. That is, up to the I picture of picture number 12 (preserved picture number 6) that should not be originally removed.

However, if the initialization command (Code5) accompanying the I picture of picture number 12 does not disappear due to transmission error by attaching the initialization retransmission command (Code6) to the P picture of picture number 14 instead of the initialization command (Code5). When the initialization command Code5 is lost due to a transmission error by the initialization command Code5, the storage picture number is 5 or less by the initialization retransmission command Code6 that accompanies the P picture of picture number 14. All pictures stored in the memory are deleted.

That is, since an initialization frame (picture number 12 in this case) is added to the initialization retransmission command Code6 as additional information, it is stored in the reference picture memory at the time of storing the storage frame (initialization frame) before the initialization frame. A storage frame having a storage picture number 5 or less) is deleted.

As described above, the initialization retransmission command Code6 having additional information increases the possibility that the initialization can be executed normally even when the initialization command Code5 is lost due to a transmission path error. In addition, in the initialization retransmission command shown in the present embodiment, substituting the additional information to the picture number accompanied by the initialization retransmission command as the initialization command, Code5 and Code6 shown in FIG. 22 may be realized by one command. do. This is because, when retransmission is performed for retransmission of the initialization information, the same picture number as that of retransmission of the frame is not used to specify the number of the frame to which the initialization command is attached. At this time, the initialization command Code5 may be invalidated.

When the initialization retransmission command and the initialization command Code5 shown in the above embodiments are realized in one command in this manner, the initialization retransmission command having additional values as special information not used in the initialization retransmission command shown in the above embodiment. It may be a command having the same function as the initialization command Code5 to be sent first.

In addition, as described in each of the above embodiments, when memory management information such as a command to open an unnecessary memory area or an initialization command is transmitted again, as shown in FIGS. Rather than being included in the header information added to the frame data and transmitted, the header information including the memory management information may be transmitted separately from the frame data. That is, the above-mentioned command to be retransmitted may not be in the same stream as the encoded picture, but may be transmitted as a separate stream, for example. It may also be recorded in a separate area of the storage medium.

In the present embodiment, when retransmitting an initialization command, a picture number (initialization frame number) of a picture that is first accompanied by an initialization command is added as additional information to the initialization retransmission command. When retransmitting a command of memory management information, such as a command indicating an open memory area or a command specifying a picture to be moved from the short-term storage memory to the long-term storage memory, the picture to be encoded is first transmitted with the command. Naturally, the picture number (information specifying the picture) may be transmitted as a parameter. By doing in this way, it is possible to detect which picture has transmitted when a transmission error occurs.

(Embodiment 9)

Next, the picture coding method and the picture decoding method according to the ninth embodiment will be described. A feature of this embodiment is to change the timing of processing based on the memory management information transmitted after the second time when the memory management information is transmitted a plurality of times.

When decoding the data repeatedly encoded by the memory management information shown in the above embodiment, the image signal necessarily accompanied by the memory management information is decoded before processing the repeatedly sent memory management information. As a specific example, the case where the command for opening an unnecessary memory area described in Embodiment 2 is transmitted a plurality of times will be described again with reference to FIG. 19.

The picture of picture number 12 shown in FIG. 19 is encoded by appending the command of Code1 shown in FIG. 22, and the picture of picture number 11 is also accompanied by coding of command of Code1. At this time, decoding is performed in accordance with FIG.

First, Code1 accompanying the picture of picture number 12 is decoded (step 110). Next, the picture of picture number 12 is decoded (step 111). Here, if Code1, which should accompany the picture of the original picture number 12, is lost in the middle of transmission (No in step 112), the processing relating to this frame ends.

The decoding process following the picture of picture number 12 in the transmission order is the picture of picture number 11.

First, Code1 coded accompanying the picture of picture number 11 is decoded (step 110). Next, the picture of picture number 11 is decoded (step 111). If this Code1 is transmitted without being lost in the middle of transmission, Code1, which is a command of memory opening, exists in the decoded memory management information (Yes in step 112), and the next processing (step 113) is reached.

Here, since the memory is not opened when the decoded picture No. 12 is decoded before the picture of Picture No. 11 is decoded (No in step 113), the memory open process is performed (Step 114).

As shown in the above embodiment, as a command to open an unnecessary memory area is transmitted a number of times, the execution of a command to be executed for a picture (picture number 12) that was not originally executed in the first command is a picture sent later. After the decoding process of the picture signal (picture number 11), a delay in command execution occurs.

Therefore, in this embodiment, the method for solving the said subject is demonstrated using FIG. 26, FIG. 27, and FIG.

Fig. 26 is a correspondence diagram showing the relationship between the memory management information and the commands used in the present embodiment.

In Fig. 26, Code indicates a command number, a command indicates the content of a command, additional information indicates additional information added to the command, and a processing position shows the timing of executing the command.

The difference from FIG. 15 is that in FIG. 26, CodeA1 to CodeA4 is a command to be executed after the decoding process of the image, while CodeA6 to CodeA9 corresponding to CodeA4 to CodeA4 is a command to be executed before the decoding process of the image.

When the memory management information is repeatedly transmitted, the command of the memory management information to be encoded first is a command (the code executed from CodeA1 to CodeA4) after the decoding (which is executed after the decoding of the image), and the repetition (after the second time). ) The command to be encoded is a command (the code A6 to CodeA9) whose processing position is before decoding (to be executed before decoding of the image).

As a result, even when the memory management information sent initially is lost, a command to be executed with the memory management information originally sent first is executed at an early time, which makes it difficult to cause a problem such as a delay.

Hereinafter, the processing sequence at the time of using the command of FIG. 26 is demonstrated using FIG. 27 and FIG.

FIG. 27 is a flowchart showing the picture coding method according to the present embodiment, and shows the operation of the picture coding device 100 shown in FIG. 5.

In Fig. 27, first, an image is encoded (step B0). After encoding, an unnecessary area (image not referred to by subsequent encoding) is examined in the memory (step B1), and it is determined whether there is an unnecessary memory area (step B2). If there is an unnecessary memory area (Yes in step B2), a command for opening the unnecessary memory area is executed after decoding the image signal, so that the memory management information for decoding is encoded (step B3). The area is opened (step B4). On the other hand, when there is no unnecessary memory area (No in step B2), the operations of step B3 and step B4 are not performed.

Next, it is determined whether the memory information control unit 101 encodes a command for opening the unnecessary memory area as memory management information accompanying the encoding of the immediately encoded image (the image before the encoding target) (step B30). If it is not encoded (No in step B30), the process ends and if it is encoded (Yes in step B30), the management information encoding unit 105 executes a command to open the unnecessary memory area before decoding the image signal. In this case, the pre-decoding memory management information is encoded (step B31), and the processing ends.

In addition, when the command accompanying the coded signal of the image encoded before in step B30 and the command to open an unnecessary memory area is encoded, the command is coded again, but it is not accompanied by the previous image. It may be accompanied by the image before an image, and the said command may be repeatedly encoded as memory management information, and may be attached to many images and transmitted.

Next, the procedure for decoding the data encoded according to the procedure of FIG. 27 will be described with reference to FIGS. 28 and 19.

FIG. 28 is a flowchart showing an image decoding method according to the present embodiment, and illustrates an operation performed by the image decoding apparatus 200 shown in FIG. 7.

In the following description, it is assumed that in Fig. 19, a picture of picture No. 12 is coded by being accompanied by a command of CodeA1 shown in Fig. 26, and coded by attaching a command of CodeA6 to a picture of picture No. 11. In FIG. 17, CodeA1 is assigned to the frame header Frm12Hdr of the frame Frm12 of the picture number 12, and CodeA6 is assigned to the frame header Frm11Hdr of the frame Frm11 of the picture number 11.

In addition, in the picture decoding apparatus, a command for opening an area of the same picture in the memory is received as long as the command is not lost due to a transmission path error. Therefore, in the picture decoding method performed by the picture decoding apparatus, even when a command for reopening an already opened picture is received, the image decoding method must determine that the received picture is correctly received.

First, the decoding process regarding the picture of picture number 12 is demonstrated.

In Fig. 28, first, the memory management information of the picture of picture number 12 is decoded (step B5), and it is checked whether the memory management information is the memory management information before decoding (step B7). Since the memory management information CodeA1 is post-decoding memory management information (No in step B7), the picture signal of picture number 12 is decoded. As described above, since the memory management information CodeA1 is the memory management information after decoding (Yes in step B9), the memory is opened (step B11), and the processing relating to the memory management information of the picture of picture number 12 ends. do.

On the other hand, when CodeA1 of the memory management information is missing, it is not determined as the memory management information before decoding in step B7 (No in step B7), and also in step B9, it is not determined as the memory management information after decoding ( No) in step B9), only the decoding of the picture signal of picture number 12 is performed (step B6), and the processing relating to the memory management information of picture number 12 ends.

Next, the decoding process regarding the frame of picture number 11 is demonstrated using FIG.

First, the memory management information of picture number 11 is decoded (step B5), and it is checked whether this memory management information is dedicated memory management information for decoding (step B7). Since CodeA6 is memory management information for decryption only (Yes in step B7), it is checked whether the memory is complete (step B8). In the process of picture number 12, if CodeA1 is executed, the memory is released (Yes in step B8), so that the image signal of picture number 11 is decoded without performing the memory open process (step B10) (step B6). . Then, it is determined whether the memory management information is for decoding (step B9). Since CodeA6 is memory management information before decoding (No in step B9), the processing relating to the memory management information of the picture of picture number 11 ends.

However, if the memory management information of picture No. 12 is lost due to a missing packet in the transfer process, and the memory is not opened in the process relating to picture number 12, the memory is completed in the process relating to picture number 11. Is determined (No in step B8), and the memory is released in the next step (step B10). After the memory is released, the picture signal of picture number 11 is decoded (step B6). Since CodeA6 is memory management information before decoding (No in step B9), the processing relating to the memory management information of the picture of picture number 11 ends.

In this way, the retransmission can be executed before the decoding of the image signal, so that even if the first sent command is lost, the execution delay of the command can be reduced.

As a specific example, the case where the memory management information is CodeA1 and CodeA6 has been explained. Even when CodeA2 and CodeA7 are used, the same processing can be realized and the same processing can be realized even when CodeA3 and CodeA8, CodeA4 and CodeA9 are used. .

It is also possible to use the initialization command CodeA5 shown in FIG. 26 as post-decryption memory management information, and to use each pair in pairs using the initialization retransmission command Code6 shown in FIG. 22 as memory management information before decoding. Do.

In addition, when one post-decoding memory management information and a plurality of decoding-only memory management information are provided as header information in one frame, the plurality of decoding-only memory management information are processed before the memory management information after decoding. Just do it.

In other words, the decoding management memory may be provided at the head of the header information shown in FIG. 17 and encoded.

The combination of the instructions shown in Figs. 29A and 29B shows whether the memory management information is the decoding management information or the decoding management information as other information, and is shown in the above embodiment. You may realize one command.

Fig. 29A is a correspondence diagram showing the contents of the command and the additional information. Fig. 29B is a correspondence diagram showing the execution timing (processing position) of the command.

30 is a schematic diagram showing a command of memory management information in header information of an encoded signal.

In FIG. 30, the coded signal of the frame FrmB has a frame header FrmBHdr, frame data such as MBa and MBb, and the frame header FrmBHdr has header information and code information CodeD.

At this time, for example, as shown in Fig. 30, the frame header FrmBHdr of the frame FrmB is displayed from the head with the code information CodeD of the command, the FlagD indicating the processing position, and the additional information indicating the additional information of the command. AddD). If there is no additional information, as shown in Fig. 30, CodeE of a command and FlagE indicating a processing position may be added to the frame header FrmBHdr. By immediately setting the Flag indicating the processing position instead of Add indicating additional information immediately after the code indicating the command, the processing in Step B7 and Step B9 shown in FIG. 28 can be optimized.

In addition, in order to distinguish whether the command execution timing is before or after the decoding of the image signal, a position before the position in the frame header of the command indicating the processing position is used, using a new command indicating the processing position of the command. Commands to be executed may be performed after decoding, and commands placed after the position in the frame header of the command indicating the processing position may be performed before decoding. As a result, when there are a large number of commands, the execution timing (processing position) of each command can be represented by one command, and the sending information is reduced compared to the case of sending a flag indicating the processing position for each command, and the coding efficiency is improved. Is improved.

A specific example will be described with reference to FIG. 31.

Fig. 31 is a schematic diagram showing a command of memory management information in header information of another coded signal.

In Fig. 31, the coded signal of the frame FrmC has a frame header FrmCHdr and frame data such as MBa and MBb, and the frame header FrmCHdr is header information. The command (dif), the command (CodeG), the additional information (AddG), and the command (CodeH) are located.

Then, it is determined whether or not the command dif indicating the processing position is in the frame header FrmCHdr, and the command CodeF before the command dif indicating the processing position is executed after decoding the frame FrmC. The command CodeG or the command CodeH which is after (dif) may be executed before decoding of the frame FrmC. In this case, if there is no command dif indicating the processing position, all the commands in the frame header FrmCHdr are executed after the decoding process of the frame FrmC.

In addition, as described in each of the above embodiments, when memory management information such as a command to open an unnecessary memory area or an initialization command is transmitted again, it is not included in the header information added to the coded signal of the image and transmitted. The header information including the memory management information may be transmitted separately from the coded signal of the image. That is, the above-mentioned command to be retransmitted may not be in the same stream as the encoded picture, but may be transmitted as a separate stream, for example. It may also be recorded in a separate area of the storage medium.

(Embodiment 10)

Next, Embodiment 10 of the present invention will be described.

In this embodiment, the unit which performs encoding differs from each said embodiment. That is, in the first embodiment, when a command for opening an unnecessary memory area is transmitted a plurality of times, the memory management information stream CtlStr and the picture coding stream VideoStr shown in FIG. 5 corresponding to the command are displayed as pictures (pictures). Although encoded in units, in the present embodiment, one frame may be encoded in units of slices as in the stream structure shown in FIG.

Encoding in slice units means that each frame has slice 1 of frame 1 having headers 1-1, CtlStr1, and VideoStr1-1, and slice 2 of frame 1 having headers 1-2, CtlStr1, and VideoStr1-2. The header, the memory management information stream CtlStr, and the picture coding stream VideoStr are encoded for each slice. After encoding by the picture coding apparatus, the picture coding apparatus outputs a data stream. A slice is a synchronization return unit, and is a band-shaped area composed of one or a plurality of blocks, and a picture is formed from a plurality of slices. The picture is a basic coding unit corresponding to one image, and the block is a basic unit of coding and decoding.

As described above, the contents when the memory management information stream CtlStr is transmitted a plurality of times are the same information in the same frame. By using the same information, addition of the memory management information stream CtlStr in the slice unit can be omitted. For example, information indicating whether a plurality of transmissions are omitted in the slice is added to the header of the slice, and "0" is indicated when the command is transmitted a plurality of times and is omitted in the slice. When transmitted in a slice (when not omitted), "1" is added. Specifically, an example is shown in FIG. 33A and will be described below. The header and the picture coding stream (VideoStr) in slice 1 to slice 3 in frame 1 are different. On the other hand, slice 1 and slice 2 have the same memory management information stream (CtlStr1), and information "1" indicating that the same memory management information stream (CtlStr1) is encoded in a plurality of slices in the same frame is slice 1 and Slice 2 has each. Slice 3 also has information " 0 " indicating that the memory management information stream CtlStr1 is omitted. Accordingly, when transmission of a plurality of times is omitted from the slice, the memory management information stream CtlStr is referred to by referring to the memory management information stream CtlStr in the slice in which "1" is shown, such as the head slice. Can be omitted, and the number of bits can be reduced.

That is, the information " 0 " indicating that the above-mentioned memory management information stream CtlStr1 is omitted is a slice (slice 3) having no information specifying the picture to be removed, and information for specifying the picture to be removed. When referencing, it is information indicating that referring to information specifying a picture of a target to be removed is referred.

The method of omitting such addition of the memory management information stream CtlStr is effective because it is unlikely that the memory management information stream CtlStr is missed many times during the transmission process.

If the presence or absence of the memory management information stream CtlStr can be determined without information indicating that the memory management information stream CtlStr is omitted, this may be omitted as shown in Fig. 33 (b). For example, when the head of the memory management information stream CtlStr can be distinguished from the head of the picture coding stream VideoStr, information indicating whether or not the memory management information stream CtlStr1 is encoded as shown in FIG. 33 (b). It is possible to confirm whether the predetermined information exists at a predetermined position from the head of each slice.

This method of omitting the addition of the memory management information stream CtlStr is effective in reducing the number of times of encoding the memory management information stream CtlStr and reducing the number of bits.

As described above, encoding has been described. Similarly, decoding of one frame can be performed in units of slices. In the second embodiment, when a command for opening an unnecessary memory area is transmitted a plurality of times, the picture decoding apparatus 200 shown in FIG. 7 and the management information stream CtlStr shown in FIG. 32 corresponding to the command are used. A stream structure having a picture coded stream (VideoStr) is separated, and each is input in picture (picture) units, and each may be input in slice units.

In the encoding and decoding according to another embodiment, one frame may be encoded and decoded in units of slices.

The encoding method and the decoding method shown in the first to the tenth embodiments described above are used in mobile communication devices such as mobile phones and car navigation systems, and in photographic devices such as digital video cameras and digital still cameras. It is possible to install by. In addition, as an installation form, three types of transmitter-only terminal and encoder-only receiving terminal other than the transmitter-and-receiver-type terminal which have both an encoder and a decoder can be considered.

(Embodiment 11)

Next, Embodiment 11 of this invention is demonstrated.

In the present embodiment, a program for realizing the configuration of the picture coding method or the picture decoding method shown in the first to the tenth embodiments is recorded in a storage medium such as a flexible disk, thereby showing in the above-described embodiments. The processing can be easily performed by an independent computer system.

FIG. 34 is an explanatory diagram in the case of performing by a computer system using a flexible disk which stores the picture coding method or the picture decoding method of the first embodiment. FIG.

Fig. 34 (b) shows the external appearance, the cross-sectional structure and the flexible disk as seen from the front of the flexible disk, and Fig. 34 (a) shows an example of the physical format of the flexible disk as the main body of the recording medium. The flexible disk FD1 is embedded in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer circumference to the inner circumference, each track being divided into 16 sectors Se in the angular direction. It is divided. Therefore, in the flexible disk which stores the said program, the image coding method as the said program is recorded in the area | region allocated on the said flexible disk FD1.

34C shows a configuration for recording and reproducing the program on the flexible disk FD1. When the program is recorded on the flexible disk FD1, the computer system Cs writes the image coding method or the image decoding method as the program through the flexible disk drive FDD. When the image coding method is constructed in the computer system by the program in the flexible disk FD1, the program is read from the flexible disk FD1 by the flexible disk drive FDD and transferred to the computer system Cs. .

In the above description, a flexible disk is used as the recording medium, but the optical disk can be used in the same manner. In addition, the recording medium is not limited thereto, and the recording medium can be similarly implemented as long as it can record a program such as an IC card, a ROM cassette, and the like.

In addition, the image coding method and the image decoding method shown in the above-described embodiments include a mobile communication device such as a mobile phone or a car navigation system, a semiconductor such as an LSI, and a recording device such as a digital video camera or a digital steel camera. It is possible to install by. In addition, as an installation form, three types of transmitter-only terminal and encoder-only receiving terminal other than the transmitter / receiver-type terminal which have both an encoder and a decoder can be considered.

Here, application examples of the picture coding method and the picture decoding method shown in the first to the tenth embodiments and a system using the same will be described.

35 is a block diagram showing an overall configuration of a content supply system ex100 that realizes a content delivery service. The area for providing communication service is divided into cells of desired size, and base stations ex107 to ex110 which are fixed wireless stations are provided in respective cells.

The content supply system ex100 is, for example, a computer ex111, a PDA (personal digital assistant) ex112 via the Internet service provider ex102, a telephone network ex104, and base stations ex107 to ex110 on the Internet ex101. ), Camera ex113, mobile phone ex114, mobile phone ex115 and the like are connected to each device.

However, the content supply system ex100 is not limited to the combination as shown in Fig. 35, and may be connected in any combination. Further, each device may be directly connected to the telephone network ex104 without passing through the base stations ex107 to ex110 which are fixed wireless stations.

The camera ex113 is a device capable of shooting video such as a digital video camera. In addition, the cellular phone is a PDC (Personal Digital Communications), CDMA (Code Division Multiple Access), W-CDMA (Wideband-Code Division Multiple Access), or GSM (Global System for Mobile Communications) GSM, or PHS (Personal Handyphone System) or the like may be any.

The streaming server ex103 is connected from the camera ex113 via the base station ex109 and the telephone network ex104, and live transmission based on the encoded data transmitted by the user using the camera ex113 is performed. It becomes possible. The encoding process of the photographed data may be performed by the camera ex113, or may be performed by a server or the like which performs the data transmission process. The moving picture data shot by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device capable of capturing still images and moving images such as digital cameras. In this case, the encoding of the moving image data may be performed by the camera ex116 or by the computer ex111. The encoding process is performed by the LSI ex117 included in the computer ex111 or the camera ex116. In addition, image encoding and decoding software may be combined with any storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium that can be read by a computer ex111 or the like. Further, the moving picture data may be transmitted to the cellular phone ex115 with the camera. The moving picture data at this time is data encoded by the LSI included in the cellular phone ex115.

In the content supply system ex100, the content (e.g., video taken of music live, etc.) shot by the user of the camera ex113, the camera ex116, etc. is encoded and processed in the same manner as in the above-described embodiment. ex103), the streaming server ex103 streams the content data to the client on request. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like, capable of decoding the encoded data. In this manner, the content supply system ex100 can receive and reproduce the encoded data at the client, and can receive and decode and reproduce the encoded data in real time at the client, thereby realizing personal broadcasting.

What is necessary is just to use the image coding method or the image decoding method which were shown in said each embodiment for encoding and decoding of each apparatus which comprises this system.

As an example, a mobile phone will be described.

Fig. 36 is a diagram showing the cell phone ex115 using the picture coding method and the picture decoding method explained in the above embodiments. The cellular phone ex115 photographed by an antenna ex201 for transmitting and receiving radio waves between the base station ex110, an image such as a CCD camera, a camera unit ex203 capable of taking still images, and a camera unit ex203. A display unit ex202 such as a liquid crystal display for displaying decoded data such as an image, an image received by the antenna ex201, a main body composed of a group of operation keys ex204, and an audio output unit such as a speaker for audio output (ex208) Encoded data or decoded data such as a voice input unit ex205 such as a microphone for voice input, data of a captured moving image or still image, data of a received mail, data of a moving image or still image The recording medium ex207 for storing the data and the slot ex206 for allowing the recording medium ex207 to be mounted on the cellular phone ex115 are provided. The recording medium ex207 stores a flash memory device which is a kind of electrically erasable and programmable read only memory (EEPROM), which is a nonvolatile memory that can be rewritten or erased in a plastic case such as an SD card.

The mobile telephone ex115 will be described with reference to FIG. 37. The cellular phone ex115 controls the main control unit ex311 configured to collectively control the respective units of the main body unit including the display unit ex202 and the operation key ex204, and includes a power supply circuit unit ex310, an operation input control unit ex304, An image coding unit ex312, a camera interface unit ex303, an LCD (Liquid Crystal Display) control unit ex302, an image decoding unit ex309, a multiple separation unit ex308, a recording / reproducing unit ex307, a modulation / demodulation circuit unit ex306 ) And the voice processing unit ex305 are interconnected via a synchronization bus ex313.

The power supply circuit unit ex310 is operable to operate the digital cellular phone ex115 with the camera by supplying power to each unit from the battery pack when the terminal and power key are turned on by a user's operation. Start up with.

The cellular phone ex115 controls the voice processing unit ex305 to collect the voice signal collected by the voice input unit ex205 in the voice call mode under the control of the main controller ex311 including the CPU, ROM, and RAM. The digital audio data is converted into digital voice data, and the spectrum demodulation process is performed by the demodulation and demodulation circuit unit ex306, and then transmitted through the antenna ex201 after the digital analog conversion process and the frequency conversion process are performed by the transmission and reception circuit unit ex301. In addition, the mobile phone ex115 amplifies the received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, and performs a spectrum despreading process by the demodulation circuit unit ex306, thereby processing the voice processing unit. After conversion to analog voice data by ex305, it is output through the voice output unit ex208.

When the electronic mail is transmitted in the data communication mode, the text data of the electronic mail input by the operation of the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spectrum spread processing on the text demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing on the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 through the antenna ex201. .

When image data is transmitted in the data communication mode, the image data shot by the camera unit ex203 is supplied to the image coding unit ex312 via the camera interface unit ex303. When the image data is not transmitted, it is also possible to display the image data shot by the camera unit ex203 directly on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312 is configured to include the picture coding device described in the present invention, by compressing and coding the picture data supplied from the camera unit ex203 by the coding method used in the picture coding device shown in the above embodiment. The data is converted into coded image data, and sent to the multiple separation unit ex308. At this time, the mobile phone ex115 simultaneously transmits the voices collected by the voice input unit ex205 while shooting with the camera unit ex203 as digital voice data to the multiplexing unit ex308 via the voice processing unit ex305. do.

The multiplexer ex308 multiplexes the encoded image data supplied from the image encoder ex312 and the audio data supplied from the voice processor ex305 in a predetermined manner, and the resulting multiplexed data is subjected to the demodulation and demodulation circuit ex306. After spread spectrum processing, the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the data through the antenna ex201.

In the case of receiving the data of the moving picture file linked to the home page or the like in the data communication mode, the demodulation circuit section ex306 performs the spectrum despreading process on the received data received from the base station ex110 via the antenna ex201, and as a result is obtained. The multiplexed data is sent to the multiple separation unit ex308.

In addition, in order to decode the multiplexed data received through the antenna ex201, the multiplexer ex308 divides the multiplexed data into a bit stream of image data and a bit stream of audio data, and the corresponding data is separated through a sync bus ex313. The coded image data is supplied to the picture decoding unit ex309, and the voice data is supplied to the voice processing unit ex305.

Next, the picture decoding unit ex309 includes the picture decoding apparatus described in the present invention, and decodes the reproduced moving picture data by decoding the bit stream of the picture data with a decoding method corresponding to the coding method shown in the above embodiments. It generates and supplies it to the display unit ex202 via the LCD control unit ex302, whereby, for example, moving image data included in the moving image file linked to the home page is displayed. At the same time, the voice processing unit ex305 converts the voice data into analog voice data, and then supplies the voice data to the voice output unit ex208. Accordingly, for example, the voice data contained in the moving picture file linked to the home page is stored. Is played.

In addition, the present invention is not limited to the example of the system. Recently, digital broadcasting by satellites and terrestrial waves has been a hot topic, and as shown in FIG. 38, the digital broadcasting system also includes at least an image encoding apparatus or an image decoding apparatus in the above embodiments. Any one of them can be combined. More specifically, a bit stream of video information is transmitted from a broadcasting station ex409 to a communication or broadcast satellite ex410 via radio waves. The broadcast satellite ex410, which has received this, transmits a radio wave for broadcasting, and receives the radio wave through an antenna ex406 of a home having a satellite broadcasting receiving facility, thereby receiving a television (receiver) ex401 or a set top box (STB) ex407. A device such as a device decodes a bit stream and reproduces it. The picture decoding apparatus shown in the above-described embodiments can also be provided in the reproducing apparatus ex403 which reads and decodes the bit stream recorded on the storage medium ex402 such as a CD or DVD as a recording medium. In this case, the reproduced video signal is displayed on the monitor ex404. In addition, a configuration in which an image decoding device is provided in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and reproduces it on the monitor ex408 of the television is also conceivable. have. At this time, the picture decoding apparatus may be combined in the television, not in the set top box. Further, a signal ex412 having an antenna ex411 is used to receive a signal from a satellite ex410, a base station ex107, or the like to reproduce a moving image on a display device such as car navigation ex413 included in the car ex412. It is also possible.

It is also possible to encode an image signal with the image encoding device shown in the above embodiment and to record it on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder for recording image signals on a DVD disk ex421 or a disk recorder for recording them on a hard disk. It is also possible to record to the SD card ex422. If the recorder ex420 includes the picture decoding apparatus shown in the above embodiments, the picture signals recorded on the DVD disk ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

In addition, the structure of the car navigation ex413 can be considered the same except the camera part ex203, the camera interface part ex303, and the image coding part ex312 among the structures shown in FIG. 37, for example. It may also be considered as a computer ex111, a television (receiver) ex401, or the like.

In addition, the terminal such as the mobile phone ex114 can be conceived of three types of installation, in addition to a transmitting / receiving terminal having both an encoder and a decoder, a transmitting terminal only for an encoder and a receiving terminal for a decoder only.

In this manner, the picture coding method or the picture decoding method shown in the above embodiment can be used for any of the above-described devices and systems, and thereby the effects described in the above embodiments can be obtained.

The present invention is not limited to the above embodiment, and various modifications or changes are possible without departing from the scope of the present invention.

The picture coding apparatus according to the present invention is useful as a picture coding apparatus provided in a personal computer having a communication function, a PDA, a broadcasting station for digital broadcasting, a mobile phone, and the like.

The picture decoding apparatus according to the present invention is also useful as a picture decoding apparatus included in a personal computer having a communication function, a PDA, an STB for receiving digital broadcasting, a mobile phone, and the like.

Fig.1 (a) is explanatory drawing of the image coding which encodes with reference to the image selected from the many reference image stored in the memory, and Fig.1 (b) is a block diagram which shows the structure of the memory in which an image is saved.

Fig. 2A is a flowchart showing a conventional picture coding method, and Fig. 2B is a flowchart showing a conventional picture decoding method.

Fig. 3A is another flow chart showing the conventional picture coding method, and Fig. 3B is another flow chart showing the conventional picture decoding method.

Fig. 4A is another flowchart showing the conventional picture coding method, and Fig. 4B is another flowchart showing the conventional picture decoding method.

5 is a block diagram showing the configuration of the picture coding apparatus of the present invention.

Fig. 6 is a flowchart showing the picture coding method according to the first embodiment of the present invention.

7 is a block diagram showing the configuration of the picture decoding apparatus of the present invention.

8 is a flowchart showing an image decoding method according to the second embodiment of the present invention.

Fig. 9 is a flowchart showing the picture coding method according to the third embodiment of the present invention.

10 is a flowchart showing an image decoding method according to Embodiment 4 of the present invention.

Fig. 11 is a flowchart showing the picture coding method according to the fifth embodiment of the present invention.

Fig. 12 is a flowchart showing the picture coding method according to the sixth embodiment of the present invention.

Fig. 13 is a flowchart showing the picture coding method according to the seventh embodiment of the present invention.

Fig. 14A is an explanatory diagram showing a relationship between a picture number, a stored picture number, and a transmission order of an image, and Fig. 14B shows a picture number to be coded, a stored picture number, and a picture number to be deleted. Fig. 14 (c) is a relationship diagram showing another relationship between a picture number to be decoded, a stored picture number and a picture number to be deleted.

Fig. 15 is a corresponding diagram showing a command of memory management information in the present invention.

Fig. 16 is a flowchart showing the command execution procedure in the eighth embodiment of the present invention.

17 is a schematic diagram illustrating a relationship between header information and frame data in the coded signal of each picture.

18 is a schematic diagram showing a command of memory management information in header information of an encoded signal.

19 is an explanatory diagram showing a relationship between a picture number, a stored picture number, and a transmission order of each image.

20 is a flowchart showing a method of encoding an initialization command.

21 is a flowchart showing a method of decoding an encoded initialization command.

Fig. 22 is a correspondence diagram showing a command of memory management information used in Embodiment 8 of the present invention.

Fig. 23 is a flowchart showing an image coding method using an initialization retransmission command in the present invention.

Fig. 24 is a flowchart showing a method of decoding the encoded initialization retransmission command in the present invention.

25 is an explanatory diagram showing another relationship between a picture number, a stored picture number, and a transmission order of each image.

Fig. 26 is a correspondence diagram showing a command of memory management information used in Embodiment 9 of the present invention.

Fig. 27 is a flowchart showing the picture coding method according to the ninth embodiment of the present invention.

Fig. 28 is a flowchart showing the image decoding method in the ninth embodiment of the present invention.

FIG. 29A is a correspondence diagram showing the contents of the command and the additional information, and FIG. 29B is a correspondence diagram showing the execution timing of the command.

30 is a schematic diagram showing a command of memory management information in header information of an encoded signal.

FIG. 31 is a schematic diagram showing a command of memory management information in header information of another coded signal. FIG.

32 is a schematic diagram illustrating a data stream structure encoded in slice units.

33A and 33B are schematic diagrams showing a data stream structure coded in slice units.

34 (a), (b), and (c) all show a storage medium for storing a program for realizing, by a computer system, the picture coding method and the picture decoding method of the first to the tenth embodiments of the present invention. It is explanatory drawing.

35 is a block diagram showing an overall configuration of a computer supply system in which the picture coding method and the picture decoding method according to the present invention are used.

36 is an external view showing an example of a mobile telephone in which the picture coding method and the picture decoding method according to the present invention are used.

37 is a block diagram showing the structure of the mobile telephone.

38 is a block diagram showing the configuration of a system for digital broadcasting in which the picture coding method and the picture decoding method according to the present invention are used.

Claims (6)

A data stream generation method for generating a data stream comprising encoded audio data, encoded information of memory management information for managing a memory holding a decoded picture as a reference picture, and an encoded picture, A first encoding step of encoding a picture to be encoded, A second encoding step of attaching and encoding memory management information to the encoded encoding target picture; A third encoding step of encoding the I picture or the P picture whose decoding order is later than the picture to be encoded; A fourth encoding step of appending the memory management information to the I picture or the P picture and encoding the same; And a fifth encoding step of encoding the audio signal. A data stream generating device for generating a data stream comprising encoded audio data, encoded information of memory management information for managing a memory for holding a decoded picture as a reference picture, and an encoded picture. First encoding means for encoding a picture to be encoded, Second encoding means for appending and encoding memory management information to the encoded encoding target picture; Third encoding means for encoding an I picture or a P picture having a decoding order after the encoding target picture; Fourth encoding means for appending the memory management information to the I picture or the P picture and encoding the same; And fifth encoding means for encoding the audio signal. A data stream decoding method for decoding a data stream generated by the data stream generating method according to claim 1, A voice decoding step of decoding the voice data; A memory management information decoding step of decoding the encoded memory management information; A picture decoding step of decoding the decoded picture; A management step of managing a reference picture stored in a memory based on the memory management information, When decoding an I picture or a P picture whose decoding order is later than the picture to be decoded, In the memory management information decoding step, the memory management information having the same content as the memory management information that is accompanied by the decoding target picture and encoded is decoded. In the picture decoding step, the I picture or the P picture is decoded. And in said managing step, managing reference pictures stored in a memory based on said memory management information. A data stream decoding device for decoding a data stream generated by the data stream generating device according to claim 2, Speech decoding means for decoding the speech data; Memory management information decoding means for decoding the encoded memory management information; Picture decoding means for decoding the picture to be decoded; Management means for managing a reference picture stored in the memory based on the memory management information; When decoding an I picture or a P picture whose decoding order is later than the picture to be decoded, The memory management information decoding means decodes memory management information having the same content as that of the memory management information that has been encoded with the decoding target picture; The picture decoding means decodes the I picture or the P picture, And said management means manages the reference pictures stored in the memory based on the memory management information.    A recording method for recording a data stream comprising encoded audio data, encoded information of memory management information for managing a memory for holding a decoded picture as a reference picture, and a data stream including the encoded picture, the recording medium comprising:    A first encoding step of encoding a picture to be encoded,    A second encoding step of attaching and encoding memory management information to the encoded encoding target picture;    A third encoding step of encoding the I picture or the P picture whose decoding order is later than the picture to be encoded;    A fourth encoding step of appending the memory management information to the I picture or the P picture and encoding the same;     A fifth encoding step of encoding the speech signal,    And a recording step of recording the data stream generated by the first to fifth encoding steps on a recording medium. A recording apparatus for recording a data stream including encoded audio data, encoded information of memory management information for managing a memory for holding a decoded picture as a reference picture, and a data stream including the encoded picture, comprising: First encoding means for encoding a picture to be encoded, Second encoding means for appending and encoding memory management information to the encoded encoding target picture; Third encoding means for encoding an I picture or a P picture having a decoding order after the encoding target picture; Fourth encoding means for appending the memory management information to the I picture or the P picture and encoding the same; Fifth encoding means for encoding an audio signal, And recording means for recording the data stream generated by the first to fifth encoding means on a recording medium.

Images