JP2009081898A - Image encoding method and image encoding apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image encoding method and image decoding method in which no trouble occurs even when a stream is switched in the middle of decoding. <P>SOLUTION: The present invention relates to a picture coding method for generating a coded signal corresponding to each picture by coding a plurality of image signals, wherein, a switching picture capable of switching a plurality of coded signals and a picture capable of referencing the switching picture after switching picture are only a group of pictures of the same time in the encoded signals. More specifically, when picture numbers are discontinuous before and after an S picture, processing is performed not to handle it as an error. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  In the present invention, an image encoding method for efficiently compressing a moving image signal using correlation between screens, an image decoding method for correctly decoding the image signal, and a program for executing the same in software are recorded. The present invention relates to a recording medium.

  In recent years, the multimedia era has come to handle voice, image, and other pixel values in an integrated manner, and conventional information media, that is, means for transmitting information such as newspapers, magazines, televisions, radios, telephones, etc., to people have become multimedia. It has come to be taken up as a target. In general, multimedia refers to not only characters but also figures, sounds, especially images, etc. that are associated with each other at the same time. It is an essential condition.

  However, when the information amount of each information medium is estimated as a digital information amount, the information amount per character is 1 to 2 bytes in the case of characters, whereas 64 kbits per second (phone quality) in the case of speech. In addition, for a moving image, an information amount of 100 Mbits (current television reception quality) or more per second is required, and it is not realistic to handle the enormous amount of information as it is in the digital format with the information medium. For example, videophones have already been put into practical use by ISDN (Integrated Services Digital Network) having a transmission rate of 64 kbps to 1.5 Mbps. Is possible.

  Therefore, what is needed is information compression technology. For example, in the case of videophones, the H.261 and H.263 standards that have been internationally standardized by the ITU-T (International Telecommunication Union Telecommunication Standardization Sector) are required. Video compression technology is used. Also, according to the MPEG-1 standard information compression technology, it is possible to put image information together with audio information on a normal music CD (compact disc).

  Here, MPEG (Moving Picture Experts Group) is an international standard for digital compression of moving picture signals, and MPEG-1 is up to 1.5 Mbps of moving picture signals, that is, about 1 / 100th of information of television signals. It is a standard that compresses up to. In addition, the MPEG-1 standard has set the target quality to a medium quality that can be realized at a transmission speed of approximately 1.5 Mbps, so that MPEG-2 has been standardized to meet the demand for higher image quality. Then, the moving image signal is compressed to 2 to 15 Mbps.

  Furthermore, MPEG-4 with higher compression ratio has been standardized by a working group (ISO / IEC JTC1 / SC29 / WG11) that has been standardizing with MPEG-1 and MPEG-2. MPEG-4 initially introduced not only high-efficiency coding at low bit rates, but also powerful error resilience technology that can reduce subjective image quality degradation even if transmission path errors occur. . Also, standardization activities of JVT (Joint Video Team) are underway as a next-generation screen coding system jointly with ISO / IEC and ITU-T, and what is currently called Joint Model 2 (JM2) is the latest .

  A picture that does not have a reference picture and performs intra prediction coding is called an I picture. A picture that performs inter-frame predictive coding with reference to only one picture is called a P picture. A picture that can be subjected to inter-picture prediction coding with reference to two pictures at the same time is called a B picture.

  Here, a picture is a term representing a single screen, which means a frame in a progressive image and a frame or field in an interlaced image. Here, an interlaced image is an image in which one frame is composed of two fields having different times. In interlaced image encoding and decoding processing, one frame may be processed as a frame, processed as two fields, or processed as a frame structure or a field structure for each block in the frame. it can.

  In JVT, unlike conventional video coding, it is possible to select an arbitrary image (picture) as a reference image from a plurality of images (pictures) as a forward reference image. In addition, a mechanism that enables switching of the encoded stream with a specific picture is introduced, and S pictures (referred to as SI pictures and SP pictures) are used for intra prediction encoding and inter prediction encoding with S pictures, respectively. Corresponding) was introduced.

  The S picture is a mechanism that guarantees that the stream after the S picture can be correctly decoded even if the stream is switched immediately before the S picture from a plurality of streams. Streams can be switched on the server according to the receiver's preference.

  Now, in such a conventional image encoding method and image decoding method, (1) an arbitrary image (picture) can be selected as a reference image from a plurality of images (pictures) as a forward reference image, (2) The S picture was introduced so that the stream could be switched with a specific picture. However, despite the introduction of the two technologies in this way, unfortunately, the problems that occur when the two are combined are not fully considered, and due to the issues described below, It was difficult to use both together.

  FIG. 29 is a diagram for explaining a correspondence between a picture and a picture number PN when the input image signal Vin is encoded. Stream 1, stream 2, and stream 3 are the same image signals encoded at different picture rates (number of pictures per second). The picture number PN is a number for identifying an encoded picture. In JM2, a reference image that is referred to in subsequent encoding is assigned a number that increases by “1”. In order to simplify the description, in the example of FIG. 29, it is assumed that all pictures in each stream are referred to in subsequent encoding, and is described only when the value of the picture number PN always increases by “1”. Pictures that are not referenced in subsequent encoding do not affect the increase or decrease of the picture number PN and are not stored in the memory. Therefore, the picture that is not referred to in the subsequent encoding does not affect the description of the future operation, and thus the description thereof is omitted.

  Now, as shown in FIG. 29, at time t3, the picture indicated by the diagonal lines of each stream is encoded as an S picture. FIG. 30 is a diagram illustrating a picture number PN of an image stored in the reference image memory Mem when an S picture is encoded and decoded.

  FIG. 30 shows an image stored in the reference image memory Mem and its position. In the reference image memory Mem, the picture at the left position has a newer time than the picture stored at the right position. When performing predictive encoding, it is necessary to refer to the same image for encoding and decoding, and when a reference image can be selected from a plurality of reference images as in JM2, which image is referred to is clearly indicated. There is a need.

  There are two ways to specify the reference image, and JM2 uses both methods according to the purpose.

(1) Specify the number of pictures before the newest time (2) Specify the picture to be referenced by picture number PN

In order to correctly encode the S picture and the picture after the S picture and to correctly decode at the time of decoding when switching the stream in the S picture, it was switched from which stream to which stream in the S picture. Even in this case, the contents of the reference image memory Mem must match.
ITU-T recommendation H.263AnnexU (11/00) "Enhanced reference picture selection mode"

  However, as is apparent from the explanatory diagram of the picture number PN of the image stored in the reference image memory Mem in FIG. 30, the contents of the reference image memory Mem match at the start of S picture encoding / decoding for each stream. Therefore, the conventional method cannot be used in combination with an encoding method for selecting a reference picture from the reference image memory Mem and an S picture mechanism for switching streams.

  Therefore, the present invention solves the above-described problems, enables the S-picture mechanism to be used in combination with an encoding method for selecting a reference picture from the reference image memory Mem, and the encoding method even when an S-picture is used. An object of the present invention is to provide an image encoding method and an image decoding method capable of improving the compression rate.

To solve this challenge,
A first invention is an image coding method for generating a coded stream of a moving picture by referring to a reference picture stored in a memory by specifying the picture number, and coding a picture number corresponding to a picture to be coded A picture number encoding step for encoding, an encoding step for encoding the encoding target picture, and after execution of the encoding step, all pictures stored in the memory other than the encoding target picture are not used. All picture non-use step, a picture number initialization step for initializing a picture number of the picture to be encoded in the memory, and a picture decoding apparatus for a picture other than the picture to be encoded already All coding that disables all picture disabling information indicating that all pictures stored in memory are to be disabled An image encoding method which comprises a puncture nonuse of information encoding step.

  A second invention is an image decoding method for decoding a moving picture encoded stream by referring to a reference picture stored in a memory by specifying a picture number, and from the encoded stream, a picture other than a decoding target picture. An all-picture disabling information decoding step for detecting and decoding all-picture disabling information, which means that all pictures already stored in the memory are not used, and from the encoded stream A decoding step for decoding the decoding target picture, and after execution of the decoding step, a picture other than the decoding target picture is already stored in the memory in accordance with the decoded all-picture disabling information For the all picture disabling step for disabling all existing pictures and the decoding target picture in the memory, An image decoding method characterized by including the picture numbers initialization step to impart reduction has been a new picture number.

  A third invention is an image encoding method for encoding a plurality of image signals and generating an encoded signal for each image, and the plurality of encoded signals can be referred to from a switching picture and a switching picture onward. A picture is an image coding method in which only the picture group at the same time in the coded signal is used.

  A fourth invention is an image decoding method for decoding an encoded signal, wherein information of a picture that is not used before a switchable switching picture is decoded, and a reference image memory is based on the result of the decoding This is an image decoding method in which a picture decoded from the above is not used, and an encoded signal is decoded by referring only to reference pictures that have not been disabled after the switching picture.

  According to a fifth aspect of the present invention, there is provided an image encoding method for encoding a plurality of image signals to generate an encoded signal for each image, wherein a picture number of a switching picture capable of switching the plurality of encoded signals is assigned to each code. This is an image coding method having a mechanism for changing to the same value by a coded signal.

  A sixth invention is an image encoding method for decoding an encoded signal, and when the encoded signal is switched by a switchable switching picture, the picture number of the reference picture is the same for the switchable encoded signal. This is an image decoding method having a mechanism for changing to a value.

  As described above, according to the image encoding method and the image decoding method according to the present invention, the S picture mechanism and the encoding method for selecting a reference picture from the reference image memory Mem can be used in combination. Even when an S picture is used, an image encoding method and an image decoding method capable of improving the compression rate in the encoding method can be provided, and their practical value is high.

  Embodiments of the present invention will be described below with reference to FIGS.

(Embodiment 1)
FIG. 1 is an explanatory diagram of picture numbers PN of images stored in the reference image memory Mem. The difference between this figure and the explanatory diagram of the picture number PN of the image stored in the reference image memory Mem of FIG. 30 will be described below.

  Conventionally, it has been described that the contents of the reference image memory Mem do not match when the encoded signal is switched when encoding / decoding an S picture. Therefore, in the encoding / decoding method of the present invention, in the correspondence explanatory diagram of the picture and the picture number PN when the input image signal Vin of FIG. 29 is encoded, the times t0, t1, Only the image of t2 is stored in the reference image memory Mem, and other images are deleted from the reference image memory Mem before encoding / decoding of the S picture. The result is an explanatory diagram of the picture number PN of the image stored in the reference image memory Mem of FIG.

  As is clear from the explanatory diagram of the picture number PN of the image stored in the reference image memory Mem in FIG. 1, when the reference image is clearly indicated by encoding / decoding, “how many pictures before the new time If the method of “specify information” is used, an image at the same time is referred to in any of stream 1, stream 2, and stream 3, so that encoding / decoding can be performed correctly.

  FIG. 2 is a flowchart of an encoding method and a decoding method of information related to image control stored in the reference image memory Mem of the image encoding method and image decoding method of the present invention.

  The flowchart of the encoding method in FIG. 2A shows a method for realizing the operation described with reference to FIG. 1 and a method for encoding / decoding information necessary for the operation.

  In Step 0, pictures having the same time are selected from a plurality of pieces of encoded information (streams). In Step 1, deletion information indicating that a picture other than that selected in Step 0 is deleted is encoded. In Step 2, pictures other than those selected in Step 0 are deleted from the reference image memory Mem. As described above, as shown in FIG. 1, the reference image memory Mem storage state for realizing a stream that can be decoded even if the encoded signal is switched can be realized.

  Note that the order of Step 1 and Step 2 may be switched, and in that case, the flowchart of the image encoding method shown in FIG.

  By decoding the deletion information encoded in the flowchart of the encoding method of FIG. 2A by the method shown in the flowchart of the decoding method of FIG. 2C, as shown in FIG. The reference image memory Mem storage state for realizing a stream that can be decoded even after switching can be realized by the image decoding method.

  By decoding the deletion information in Step 5, it is possible to clearly indicate pictures that do not have the same time among a plurality of pieces of encoded information (streams). This corresponds to a picture other than the picture having the same time selected in Step 0 of FIG. Next, in Step 6, the picture selected in Step 5 is deleted from the reference image memory Mem. Specifically, when deleting (or deleting) a picture stored in the reference image memory Mem, identification information such as “not used” that prohibits the picture to be deleted from being used as a reference image. Do by setting. Correspondingly, whether the image decoding unit PicDec and the image encoding unit PicEnc always have “not used” identification information set when referring to a picture stored in the reference image memory Mem. After confirming whether or not the “unused” identification information is set, the picture is not referred to, and only the picture for which the “unused” identification information is not set is referred to. In the following embodiments, the pictures in the reference image memory Mem are deleted (or deleted) in the same manner. Of course, this deletion method is an example, and it goes without saying that the picture data may be deleted by actually deleting (or deleting) the data from the reference image memory Mem. As described above, even when the encoded signal is switched, the reference image memory Mem storage state for realizing a decodable stream as shown in FIG. 1 can be realized.

(Embodiment 2)
FIG. 3A is an explanatory diagram of a picture number PN of an image stored in the reference image memory Mem. The difference between the picture numbers PN of the images stored in the reference image memory Mem in FIG. 3A and FIG. 1 is whether or not the picture numbers PN in the reference image memory Mem match.

  Not only the time of the picture stored in the reference image memory Mem but also the picture number PN in the reference image memory Mem is matched in all the encoded signals, so that the reference image is clearly indicated by encoding / decoding. In this case, a method of “specifying a picture to be referred to by a picture number PN” becomes available, and an image at the same time is referred to in any of the stream 1, the stream 2, and the stream 3. -Decoding can be performed correctly.

  To realize this, before encoding / decoding an S picture, the picture number PN of the picture stored in the reference picture memory Mem is changed to the same value, and the information for the change is encoded.・ Decryption is enough.

  Further, since the next S picture must be saved as the same picture number PN, the picture number PN of the S picture must be matched in any stream.

  FIG. 4 is a flowchart of an encoding method and a decoding method for information related to image control stored in the reference image memory Mem of the image encoding method and the image decoding method of the present invention, and will be described with reference to FIG. A method for realizing the above operation and a method for encoding / decoding information necessary for the operation will be described.

  In Step 10, the maximum value (8 in the example of FIG. 3A) of the picture number PN of the picture included in the reference image memory Mem is detected from the encoded signal to be switched. In Step 12, information for reassigning the picture number PN of each picture stored in the reference image memory Mem is encoded based on the maximum value of the picture number PN. Further, if necessary, a picture number PN to be assigned to the next S picture is also encoded. Since stream 3 in FIG. 3A and stream 3 in FIG. 1 are the same, there is no need to reassign stream 3. Therefore, only necessary reassignment of the picture number PN is performed, and only necessary reassignment information is encoded in Step 11. Finally, reassignment of the picture number PN indicated by the information encoded in Step 11 is performed in Step 12. As described above, as shown in FIG. 3, the reference image memory Mem storage state for realizing a stream that can be decoded even if the encoded signal is switched can be realized.

  Also, since the picture number PN of the S picture is 12, in order to make the picture numbers PN consecutive after the encoding / decoding of the S picture, as shown in FIG. 11 which is the number PN (immediately before the S picture of stream 1 in FIG. 29) may be used. In this case, since the picture number PN of the S picture is 12, the picture number PN always increases during the encoding / decoding process, and an error detection function that makes an error when the picture number PN decreases is realized. Can be more effective.

  FIG. 5 is a diagram for explaining the correspondence between a picture and a picture number PN when the input image signal Vin of the present invention is encoded. FIG. 5 shows an example in which the picture number PN is reassigned by the method described with reference to FIG. 3B. In the S picture, the picture number PN is all 12, which is referred to when encoding / decoding the S picture. If the image in the image memory Mem is constant regardless of the stream, it is clear that all images after the S picture can be correctly decoded even if the stream is switched with the S picture.

  Note that the order of Step 11 and Step 12 may be interchanged, and in that case, the flowchart of the image encoding method shown in FIG.

  The deletion information encoded in the flowchart of the encoding method in FIG. 4A is decoded by the method shown in the flowchart of the decoding method in FIG. The reference image memory Mem storage state for realizing a stream that can be decoded even when the conversion signal is switched can be realized by the image decoding method.

  By decoding the reassignment information of the picture number PN in step 15, it is possible to specify an image that needs to be reassigned the picture number PN and its method. Next, in Step 16, the picture number PN in the reference picture memory Mem is reassigned based on the image decoded in Step 15 and requiring the picture number PN reassignment. As described above, even if the encoded signal is switched, the reference image memory Mem storage state for realizing a decodable stream can be realized as shown in FIG.

  In the present embodiment, the effectiveness in combination with the first embodiment has been described. However, the advantage of being able to correctly encode / decode when “pictures to be referred to by the picture number PN” are implemented is implemented. If only this effect is sufficient, it can be used without being combined with the first embodiment.

(Embodiment 3)
FIG. 6 shows another embodiment for realizing the explanatory diagram of the picture number PN of the image stored in the reference image memory Mem of FIG.

  The picture type is identified by picture type information PicType. Therefore, if the picture type information PicType is an S picture in which the stream can be switched, the reference picture memory is determined by determining a rule for reassigning the picture number PN of the reference picture memory Mem so as to match the picture number PN of the S picture. It is possible to omit the encoding / decoding of information on the method of reassigning individual picture numbers of Mem.

  Hereinafter, the operation of FIG. 6A will be described. In Step 20, the encoded signal is decoded to obtain the picture number PN of the image. If the picture type information PicType of the image acquired in Step 21 is acquired and the picture type information PicType is an S picture, a picture in the reference image memory Mem is used to match the picture number PN of the S picture using a predetermined method in Step 22. Reassign number PN. As described above, as shown in FIG. 3, the reference image memory Mem storage state for realizing a stream that can be decoded even if the encoded signal is switched can be realized.

  Note that the order of Step 21 and Step 22 may be switched, and in that case, the flowchart of the image encoding method shown in FIG.

  6 and FIG. 4 are combined with explanatory diagrams of picture numbers PN of images stored in the reference picture memory Mem of FIG. 3, and a part of the reassignment information of picture numbers PN in Step 11 and Step 15 of FIG. Only those that cannot be expressed by the rule of reassigning the picture number PN of the reference picture memory Mem so as to match the picture number PN of the picture may be encoded / decoded.

(Embodiment 4)
FIG. 7 is a block diagram showing the configuration of the image coding apparatus of the present invention. The block diagram of the image coding apparatus of the present invention in FIG. 7 is an example for realizing the image coding method according to the first and second embodiments.

  The picture number generation unit PNGen generates a picture number PN. The picture number PN is an identifier for distinguishing images stored in the reference image memory Mem, and different pictures stored in the reference image memory Mem are assigned different picture numbers PN. Normally, every time an image is stored in the reference image memory Mem, the picture number PN is incremented by 1. If the picture number PN received by the image decoding apparatus is increased by 2 or more, an image to be stored is lost due to a transmission path error. This can be detected by an image decoding device and error correction or error correction processing can be performed.

  The maximum picture number detection unit MaxPN compares the other encoded signal picture number OtherPN and the picture number PN generated by the picture number generation unit PNGen, detects the maximum value of the picture number PN, and notifies the variable length encoding unit VLC At the same time, the picture number generation unit PNGen is notified and the picture number PN generated by the picture number generation unit PNGen is initialized with the maximum value of the picture number PN. The other encoded signal picture number OtherPN is a picture number of a picture of a different stream corresponding to the same encoding target picture. As a result, the picture number generation unit PNGen thereafter outputs a picture number PN larger than the maximum value of the picture number PN.

  The encoded picture time comparison unit TimeCmp compares the time of each picture of the input image signal Vin encoded so far with the picture time FrameTime of each picture encoded as another encoded signal (stream), and performs all encoding. Information of the picture at the time encoded with the signal is notified to the image removal unit PicDel.

  When the picture type information PicType indicates that the next picture is an S picture, the image removal unit PicDel is encoded with all the encoded signals based on the information notified from the encoded picture time comparison unit TimeCmp. A command to erase an image stored in the reference image memory Mem other than the picture at the current time is notified to the reference image memory Mem, and at the same time, the information is also notified to the variable length coding unit VLC.

  The image encoding unit PicEnc refers to the image stored in the reference image memory Mem, encodes the input image signal Vin as a picture type indicated by the picture type information PicType with frequency conversion, quantization, etc., and decodes the result. Is transmitted to the encoding unit PicDec and the variable length encoding unit VLC. The image decoding unit PicDec performs inverse quantization / inverse frequency conversion on the result encoded by the image encoding unit PicEnc as a picture type indicated by the picture type information PicType, and as a picture number PN for reference in subsequent image encoding Save to reference image memory Mem.

  The variable length coding unit VLC performs variable length coding on the result of coding by the image coding unit PicEnc to form a bit string, and reference image memory Mem notified from the image removal unit PicDel, which is information necessary for decoding The information for erasing the image stored in, the maximum value of the picture number PN and the picture number PN are encoded and output as an encoded signal Str. Further, based on the information notified from the image removal unit PicDel and the picture number PN, the information for changing the picture number PN of the image stored in the reference image memory Mem is also encoded based on the method described in the second embodiment. .

  FIG. 8 shows a configuration example of the encoded signal Str of the present invention. Hereinafter, each data of FIG. 8A will be described.

  First, the picture number PN is encoded. Next, the maximum PN to be replaced, information for deleting the image stored in the reference image memory Mem, and information for replacing the picture number PN of the image stored in the reference image memory Mem are encoded. Subsequently, picture type information PicType and picture coded data which is an output of the picture coding unit PicEnc are arranged.

  FIG. 8A is merely an example of data arrangement, and can be realized by changing the order of data as shown in FIG. 8B.

  With the above configuration, it is possible to realize an image encoding apparatus that realizes the image encoding method according to the first and second embodiments.

(Embodiment 5)
FIG. 9 is a block diagram showing the configuration of the image decoding apparatus of the present invention. The block diagram of the image decoding apparatus of the present invention in FIG. 9 is an example of an image decoding apparatus that realizes the first embodiment, the second embodiment, and the third embodiment. The operation will be described below.

  The variable length decoding unit VLD decodes the encoded signal Str, and includes various information (a command for deleting an image stored in the reference image memory Mem, picture type information PicType, picture number PN, information for changing the picture number PN, and Image data).

  First, an instruction to delete an image stored in the reference image memory Mem obtained by the variable length decoding unit VLD is notified to the image removal unit PicDel, and the image removal unit PicDel is stored in the reference image memory Mem. Erase the commanded image.

  The picture type information PicType obtained by the variable length decoding unit VLD is notified to the image decoding unit PicDec and indicates the decoding method.

  The picture number PN obtained by the variable-length decoding unit VLD is notified to the reference image memory Mem, and is used as the picture number PN for storing the image decoded by the image decoding unit PicDec.

  The information for changing the picture number PN of the image stored in the reference image memory Mem obtained by the variable length decoding unit VLD is notified to the picture number change unit PNchg, and the picture number change unit PNchg follows the instruction, and the reference image memory Change the picture number PN of the image stored in Mem. More specifically, the picture number changing unit PNchg reads the picture number PN of the image stored in the reference image memory Mem, changes the value of the read picture number PN, and then refers to the picture number PN Write to image memory Mem.

  The image data obtained by the variable length decoding unit VLD is decoded by the image decoding unit PicDec by a decoding method according to the picture type indicated by the picture type information PicType. That is, the I picture is decoded without referring to the image in the reference image memory Mem, and the P picture and the B picture are decoded with reference to the image stored in the reference image memory Mem. The decoded image obtained in this way is stored in the reference image memory Mem and is output as a decoded image signal Vout.

  As described above, the image decoding apparatus that realizes the first embodiment, the second embodiment, and the third embodiment can be realized.

(Embodiment 6)
In the image coding apparatus described in Embodiments 1 to 5, when a stream is switched with an S picture, the picture number of the picture preceding the switchable picture is continuous with the picture number of the switchable picture. Was supposed to be switched. In the present embodiment, picture numbers are switched in a switchable picture.

  As an example of stream switching, regarding the encoding of a plurality of streams having the same picture with different picture rates, different bit rates, and different picture structures, pictures belonging to the other streams from the stream to which the picture that is being encoded belongs This is a process of proceeding to encoding by switching to. The following is simply referred to as stream switching for convenience of explanation.

  Furthermore, in the present embodiment, whether or not to store the encoding target picture in the reference memory is determined by whether or not the adjacent picture (stream order) in the encoding order (stream order) of the encoding target picture in each stream. Hereinafter, the determination is made according to the degree of change between the picture number of the previous picture) and the picture number to be encoded. Specifically, if the picture number of the picture to be encoded is increased by “1” with respect to the picture number of the previous picture, this indicates that the picture to be encoded is stored in the reference memory. On the other hand, if the picture number of the encoding target picture is the same as the previous picture number with respect to the picture number of the previous picture, it indicates that the encoding target picture is not stored in the reference memory.

  Hereinafter, processing for switching picture numbers in a switchable picture will be described in detail with reference to the drawings.

  FIG. 10 is a diagram illustrating an example of a correspondence between a picture and a picture number PN when the input image signal Vin is encoded. Each of stream 1, stream 2, and stream 3 is obtained by encoding the same image signal at different picture rates. In FIG. 10, pictures are arranged for each stream in the order of encoding.

  In the stream 1, a picture number PN is assigned to each picture so as to increase by “1” between the pictures. In stream 2, there are a picture to which a picture number is assigned so as to increase by “1” between pictures and a picture to which the same picture number as the previous picture is assigned. Further, in the stream 3, like the stream 1, the picture number PN is assigned to each picture so as to increase by “1” between the pictures.

  Therefore, in the streams 1 and 3, since the picture number increases by “1” between the pictures, the picture to be encoded is stored in the reference memory. In stream 2, a picture assigned a picture number so as to increase by “1” between pictures is stored in the reference memory, and a picture assigned the same picture number as the previous picture is not saved.

  Also, the picture with the picture number “0” in the stream 1, the picture with the picture number “0” in the stream 2, and the picture with the picture number “0” in the stream 3 correspond at time t0. Similarly, picture F14 of stream 1, picture F22 of stream 2 and picture F31 of stream 3 are at time t1, picture F18 of stream 1, picture F24 of stream 2 and picture F32 of stream 3 are at time t2, stream 1 Picture F112, stream 2 picture F26, and stream 3 picture F33 correspond to time t3, and stream 1 picture F117, stream 2 picture F215, and stream 3 picture F34 correspond to time t4. Note that the pictures F112, F26, and F33 correspond to the S pictures in the first and second embodiments.

  In FIG. 10, pictures BP1 and BP2 are passed when the streams are switched, and are the same as the pictures before the switching destination picture in the stream to which the switching destination picture belongs between the switching source picture and the switching destination picture. It is a switch picture encoded corresponding to time.

  For example, when the stream is switched from the picture FO26 (switching source picture) of the stream 2 to the picture F113 (switching destination picture) of the stream 1, the switch picture BP1 is used as the picture at time t3 between the picture F24 and the picture F113. . In this case, the picture number of the switch picture BP1, which is a switching picture, is changed to “12” so as to be continuous with the picture number “13” of the switching destination picture F113.

  Similarly, when the stream is switched from the picture F32 (switching source picture) of the stream 3 to the picture F213 (switching destination picture) of the stream 2, the switch picture is used as the picture at time t3 between the picture F32 and the picture F213. Use BP2. In this case, the picture number of the switch picture BP2 that is the switching picture is changed to be continuous with the picture number “13” of the switching destination picture F213.

  In this way, by assigning the picture number of the switch picture so as to be continuous with the picture number of the picture to be switched to, the picture of the picture to be switched between when encoding according to the flow for each stream and when the stream is switched The numbers are the same value.

  Next, the flow of picture number assignment processing when switching streams will be described.

  FIG. 11 is a flowchart showing a method of encoding by assigning a picture number to each picture of the stream of FIG.

  In step 1401, it is determined whether the current picture is an S picture. If the encoding target picture is an S picture, the picture number of the encoding target picture is changed to an initial value M in step 1402. If the encoding target picture is not an S picture, the picture number of the encoding target picture is not changed.

  In step 1403, it is determined whether or not the current picture to be encoded is the next picture after the S picture. If the picture to be encoded is a picture next to the S picture, it is determined in step 1404 whether the S picture is stored in the memory. If the encoding target picture is not the next picture of the S picture, it is determined in step 1405 whether or not the encoding target picture is stored in the memory.

  If it is determined in step 1404 that the S picture is stored in the memory, the picture number M is incremented to M + 1 in step 1406, and the incremented picture number becomes a new picture number.

  If it is determined in step 1404 that the S picture is not stored in the memory, the picture number is set to M itself in step 1407 (the picture number is not changed). In step 1405, it is determined whether the picture to be encoded is stored in the memory. If it is determined that the picture to be encoded is stored in the memory, the picture number PN is incremented to PN + 1 in step 1408, and the incremented picture number becomes a new picture number.

  When it is determined that the picture to be encoded is not stored in the memory, the picture number is not changed.

  In step 1409, the target picture is encoded. In step 1410, it is determined whether all the encoding target pictures have been encoded. When all the encoding target pictures have not been encoded, the process returns to step 1401. When all the encoding target pictures have been encoded, the process is terminated.

  By the process shown in FIG. 11, an encoded data stream in which picture numbers are continuous in pictures after the switch picture can be generated.

  The encoded signal Str generated in this way may be decoded according to the decoding device in the image decoding device of the fifth embodiment. In this way, an image decoding apparatus that decodes the encoded signal in the present embodiment can be realized.

  The encoding method and decoding method described in the first to sixth embodiments are applied to mobile communication devices such as mobile phones and car navigation systems, and photographing devices such as digital video cameras and digital still cameras. It can be mounted by a semiconductor such as an LSI. In addition to the transmission / reception type terminal having both an encoder and a decoder, there are three possible mounting formats: a transmitting terminal having only an encoder and a receiving terminal having only a decoder.

(Embodiment 7)
The picture referred to by the decoding target picture is specified by the picture number PN. Further, the error of the picture number PN can be detected by increasing or decreasing the picture number PN. FIG. 12 shows a procedure for detecting and correcting an error of the picture number PN based on the picture number PN in the picture data shown in FIG. 8B, for example.

  First, in Step 20, the picture number PN is detected. Next, in Step 21, picture type information PicType is detected. Then, it is determined whether or not the picture numbers PN detected in Step A2 are continuous. If the picture numbers PN are consecutive in Step A2, the error detection / correction processing for the picture numbers PN is terminated. On the other hand, if the picture numbers PN are not consecutive in Step A2, error correction is performed in Step A3. The processing such as maximum stored PN detection and PN reassignment shown in FIG. 4 may be performed after the error detection / correction processing is completed, or may be performed in parallel with the error detection / correction processing. Good.

  As a first method related to the error correction process in Step A3, a request is made to retransmit the data relating to the picture number PN in which the error has occurred, and the process according to the procedure for detecting the error of the picture number PN again after receiving the retransmission is performed. Can be considered. However, the discontinuity of the picture number PN in the S picture is not caused by a transmission error. In other words, since the discontinuity of the picture number PN in the S picture has a possibility that the number of pictures stored in the memory before the S picture differs for each stream, a retransmission of data relating to the picture number PN is requested. However, there is a possibility that the corresponding picture does not exist, and there is a great possibility that the picture corresponding to the discontinuous picture cannot be retransmitted. Therefore, since a picture that is not retransmitted is continuously requested until it is retransmitted, there is a possibility that the image cannot be reproduced. A solution to an example in which the image cannot be reproduced successfully for this reason will be described in detail in the following tenth embodiment.

  Also, when the stream is switched, if the number of pictures in the memory corresponding to the stream to be switched to does not completely match the number of pictures in the memory corresponding to the stream when switching is not performed, the image is played back. There is a possibility that it cannot be done correctly.

  First, as shown in FIG. 13, the memory stores a short-time storage memory, which is a first-in first-out memory, and direct storage location designation instead of first-in first-out in order to store pictures for a longer time than the short-time storage memory. A long-term storage memory. When the short-time storage memory is large enough to store 7 pictures and the long-time storage memory can store 4 pictures, the picture to be referenced is counted from the short-time storage memory among the pictures in the memory. Specify whether the picture is in the second. For example, usedLT2 in the long-time storage memory can be said to be the eighth picture (Idx = 7). In this way, the reference picture is designated by the relative positional relationship.

  In addition, when there are three streams as shown in FIG. 5, the position on the memory for designating the same picture (for example, the S picture shown in FIG. 5) as shown in FIG. 30 differs depending on each stream. When a picture of another stream is referred to by the S picture, the position on the memory for designating the reference picture differs depending on the memory for each stream. An S picture has a plurality of streams, and when moving from a predetermined stream to another stream, a picture that is predictively encoded with reference to a picture preceding the S picture of the stream before the movement, A picture that is the same image as a picture that is predictively encoded with reference to a picture preceding the S picture of the previous stream. (FIG. 10).

  In addition, even in the case of an I picture instead of an S picture, if the images decoded by a plurality of streams in the reference memory completely match, the streams can be switched, so that the I picture is used for the same purpose as the S picture (stream Switching).

  If various conditions are taken into consideration, it is difficult to specify the picture to be referenced accurately when the number of pictures in the memory does not match, and there is a high possibility that an error will occur even if the picture to be referenced is specified.

  Therefore, in the present embodiment, a coding method and a decoding method for additional information used to prevent the error detection process for the picture number PN from ending due to problems such as discontinuity in the picture number PN and mismatch in the memory contents. Will be described. This additional information (all-picture deletion information) is an I picture that is to be encoded so that no error occurs in the encoding process of the I picture that is subjected to intra-picture encoding or the picture after the S picture is encoded. This is an instruction indicating that all the pictures other than the S picture are deleted from the memory for reference in encoding or decoding.

  As a result, after moving from a predetermined stream to another stream, the memory state of each of the plurality of streams becomes the same. Therefore, even when a reference picture is required for inter-picture predictive coding, the predetermined picture is stored in the memory. Can be specified accurately. In addition, when moving from a predetermined stream to another stream, the fact that the picture numbers are not consecutive is not corrected as an error, thereby making it impossible to perform decoding by requesting retransmission of an image that does not actually exist. Can also be eliminated.

  Hereinafter, the encoding method will be described with reference to FIG. FIG. 14A shows a procedure for creating an encoded signal in the present embodiment.

  First, in Step 20, the picture number PN is detected. Next, in Step 21, picture type information PicType is detected. In Step A1, it is determined whether or not the detected picture type is an I picture. If the detected picture type is an I picture, in Step A10, all the pictures on the memory other than the I picture to be encoded are deleted. Subsequently, in Step A11, all picture deletion information which means that all pictures on the memory are deleted is encoded, and the additional information encoding procedure is terminated.

  As shown in FIG. 14B, the same encoding process can be performed when Step A1 in FIG. 14A is determined as a step of determining whether or not the picture type is an S picture. Further, Step A1 and Step A2 may be combined to determine whether the picture is an I picture or an S picture after the picture type is detected in Step 21.

  Also, as shown in FIG. 15A, the picture type is detected in Step 21, and if the current picture is an I picture in Step A1, it is determined whether the picture numbers PN are consecutive as in Step A3. Then, when the picture numbers PN are not continuous, all the pictures on the memory other than the I picture to be encoded may be deleted. On the other hand, if the picture numbers PN are consecutive in Step A3, the picture in the memory is not deleted. Even when an S picture is detected as the picture type, the same explanation as in FIG. Further, Step A1 and Step A2 may be combined to determine whether the picture is an I picture or an S picture after the picture type is detected in Step 21.

  After step A3 shown in FIG. 15 (a), as shown in FIG. 15 (b), step A30 is processed to determine whether or not the number of pictures stored in the memory is the same. However, the number of pictures stored in the memory may be different to prevent an error from occurring. Further, before the process of Step A3 in FIG. 15B, the process of Step A30 is performed. If the number of pictures is not the same between the streams, the process of deleting all the pictures in Step A10 is performed, and the number of pictures is the same between the streams. If it is present and the picture numbers are not consecutive, the process of deleting all pictures in Step A10 may be performed (FIG. 16).

  As described above, according to the procedure shown in FIG. 15, it is possible to leave as much as possible a picture that may become a reference image in the memory, and to improve the reproducibility of the image while reducing errors. Also, when the number of pictures stored in the memory is different in the I picture or S picture, or when the picture numbers are not consecutive, error correction processing is not required, so that memory management in the encoding device is simplified. Can be

  Further, the fact that it is an I picture and deletion of all the pictures in the memory may be indicated by a picture type indicating a special I picture.

(Embodiment 8)
FIG. 17 is a block diagram showing the configuration of the image coding apparatus of the present invention. The block diagram of the image coding apparatus of the present invention shown in FIG. 17 is an example for realizing the image coding method of FIG.

  The picture number generation unit PNGen generates a picture number PN. The picture number PN is an identifier for distinguishing images stored in the reference image memory Mem, and different pictures stored in the reference image memory Mem are assigned different picture numbers PN. Normally, every time an image is stored in the reference image memory Mem, the picture number PN is incremented by “1”, and when the picture number PN received by the image decoding apparatus increases by “2” or more, it should be stored as a transmission path error. It is now possible to perform error correction processing such as error correction (making errors inconspicuous) or error correction (playing pictures with no error by retransmission) by detecting that an image has been lost. ing.

  The image removal unit PicDel3 erases the image stored in the reference image memory Mem other than the picture to be encoded when the picture type information PicType indicates an S picture (corresponding to the processing of Step A2 in FIG. 14). To the reference image memory Mem, and at the same time, notifies the variable length coding unit VLC of the information.

  Alternatively, when the picture removal unit PicDel3 indicates that the picture type information PicType is an I picture (corresponding to the processing of Step A1 in FIG. 14), the image stored in the reference image memory Mem other than the encoding target picture Is notified to the reference image memory Mem, and at the same time, the information is also notified to the variable length coding unit VLC.

  The image encoding unit PicEnc refers to the image stored in the reference image memory Mem, encodes the input image signal Vin as a picture type indicated by the picture type information PicType with frequency conversion, quantization, etc., and decodes the result. Is transmitted to the encoding unit PicDec and the variable length encoding unit VLC.

  The image decoding unit PicDec performs inverse quantization / inverse frequency conversion on the result encoded by the image encoding unit PicEnc as a picture type indicated by the picture type information PicType, and as a picture number PN for reference in subsequent image encoding Save to reference image memory Mem.

  The variable-length encoding unit VLC performs variable-length encoding on the result encoded by the image encoding unit PicEnc to form a bit string, and the reference image memory Mem notified from the image removal unit PicDel3, which is information necessary for decoding The information for erasing the image stored in the picture, the picture number PN, and the picture type information PicType are encoded and output as an encoded signal Str.

  FIG. 8C and FIG. 8D show a configuration example of the encoded signal Str of the present invention. Each data will be described below.

  First, the picture number PN is encoded. Next, information for erasing the image stored in the reference image memory Mem, followed by picture type information PicType and image encoded data output from the image encoding unit PicEnc are arranged.

  Note that FIG. 8C is merely an example of data arrangement, and the data order can be changed as shown in FIG. 8D.

  With the above configuration, an image encoding device that realizes the image encoding method shown in FIG. 14 can be realized. Also, it is possible to provide an encoding device with high error tolerance.

(Embodiment 9)
FIG. 18 is a block diagram showing the configuration of the image encoding apparatus of the present invention. The block diagram of the image coding apparatus of the present invention in FIG. 18 shows an example for realizing the image coding method in FIG. In the following description, description of the same unit as that in FIG. 17 is omitted.

  FIG. 18 differs from FIG. 17 in the processing in the image removal unit PicDel4. Specifically, the image removal unit PicDel4 shows that the picture type information PicType is an S picture (corresponding to the processing of Step A2 in FIG. 15), and the number of pictures in the memory is not the same when compared (FIG. 15). 15 corresponding to the processing of Step A30), a command to erase the image stored in the reference image memory Mem other than the picture to be encoded is notified to the reference image memory Mem, and at the same time, the variable length coding unit VLC Notify information. The same applies when the picture type information PicType is an I picture. The encoded signal of the present invention has the same configuration as that shown in FIGS. 8 (c) and 8 (d).

  With the above configuration, an image encoding device that realizes the image encoding method shown in FIG. 15 can be realized. Also, it is possible to provide an encoding device with high error tolerance.

(Embodiment 10)
In the seventh embodiment, when the discontinuity of the picture number PN occurs in the S picture, it is shown that there is a possibility that the image cannot be reproduced because it continues to request a picture that is not retransmitted until it is retransmitted. Hereinafter, a solution for an example in which an image cannot be reproduced successfully for this reason will be described.

FIG. 19A shows a procedure for decoding an encoded signal.
First, in Step 20, the picture number PN is detected. Next, in Step 21, picture type information PicType is detected. In Step A1, it is determined whether or not the detected picture type is an I picture. If the detected picture type is not an I picture, it is determined whether or not picture numbers PN are consecutive in Step A3. On the other hand, if the detected picture type is an I picture, a series of processing ends without performing error detection / correction processing.

  If the picture numbers PN are not consecutive in Step A3, error correction is performed in Step A4. On the other hand, if the picture numbers PN are consecutive in Step A3, the error detection process is terminated.

  The error correction in Step A4 is, for example, all-picture deletion information that means that all the pictures in the memory are deleted even in the processing such as maximum stored PN detection and PN reassignment described in the above embodiment. In response, all the pictures on the memory may be deleted.

  As shown in FIG. 19B, the same encoding process can be performed even when Step A1 of FIG. 19A is used as a step of determining whether or not the picture type is an S picture. Further, Step A1 and Step A2 may be combined to determine whether the picture is an I picture or an S picture after the picture type is detected in Step 21.

  As described above, when the picture numbers are not consecutive in the I picture or the S picture, it is possible to prevent the retransmission request from being repeated and being unable to be decoded for error correction. Note that the processing in this I picture is particularly effective in the case of a special I picture that can switch streams.

(Embodiment 11)
FIG. 20 is a block diagram showing the configuration of the image decoding apparatus of the present invention. The block diagram of the image decoding apparatus of the present invention in FIG. 20 shows an example for realizing the image decoding method in FIG. In the following description, description of the same unit as that in FIG. 9 is omitted.

  20 differs from FIG. 9 in the processing in the error detection unit ErrChk based on the PN continuation determination unit PNchk and the picture type information PicType. Specifically, if the picture numbers PN input to the PN continuity determination unit PNchk are not consecutive and the picture type information PicType is not an I picture or an S picture, an error correction instruction Err is output from the error detection unit ErrChk. . If there is an error correction instruction Err, for example, processing such as maximum stored PN detection or PN reassignment, or all-picture deletion information that means deleting all the pictures on the memory, Process to delete all the pictures.

  With the above configuration, an image decoding apparatus that realizes the image decoding method shown in FIG. 19 can be realized. Also, it is possible to provide a decoding device with high error tolerance.

(Embodiment 12)
In the present embodiment, another method is described that can prevent the error detection process for the picture number PN from ending due to problems such as discontinuity of the picture number PN and mismatching of the memory contents. This embodiment is different from the seventh embodiment in that there is a step of deleting all pictures in the encoding process in the seventh embodiment. However, when deleting all the pictures, the picture number is reset from “0”. Is to assign.

  Thereby, after moving from a predetermined stream to another stream, the memory state of each of the plurality of streams becomes the same, and the picture number is initialized. Therefore, a reference picture is required for inter-picture predictive coding or the like. Even in this case, a predetermined picture can be accurately specified on the memory. In addition, when the encoded stream to be decoded is moved (switched) from a predetermined stream to another stream, the fact that the picture numbers are not consecutive is not corrected as an error, thereby making it impossible to perform decoding. Can be resolved.

  As already described, in an encoded stream obtained by encoding a moving image, a picture number PN that is consecutive in order of display time is assigned to each picture in the stream. The reason why the picture numbers PN are given in order of the display time of each picture is that when the image decoding apparatus receives the encoded stream via the transmission path, the picture in the encoded stream has a transmission error. This is because it is possible to detect the loss due to the above. When such a picture decoding apparatus decodes a received encoded stream and the picture number PN between pictures input in the order of display time increases by 2 or more, a subsequent picture is received after receiving the previous picture. It is possible to detect that there has been a transmission error until the picture is received and request the transmission side to retransmit the missing picture. Therefore, as long as the image decoding apparatus continues to decode one encoded stream, it effectively detects a transmission error in this way, receives a retransmission of a missing picture, and decodes a complete encoded stream. Can be

  However, a plurality of encoded streams obtained by encoding the same moving image at different picture rates are input, and one encoded stream is switched to another encoded stream having a different picture rate during decoding. In an image decoding apparatus that continues decoding, such error detection is conversely causing a problem that the error detection process for the picture number PN does not end. This is because the picture number PN of each picture in each encoded stream is only consecutive in the order of display time, and even between pictures that should be displayed at the same time between encoded streams with different picture rates. This is because the picture numbers PN other than the picture do not match. Therefore, in the image decoding apparatus, when the decoding target is switched to another encoded stream in the middle of decoding one encoded stream, the picture numbers PN are discontinuous even for pictures displayed at the same time. Become. Thus, in order to avoid the error detection process for picture number PN from ending due to problems such as discontinuity in picture number PN and mismatch in memory contents, additional information (all picture deletion information) is used in the seventh embodiment. The encoding method to be used has been described. This additional information is a picture other than the encoding target so that an error at the time of stream switching does not occur in the encoding process of the picture after encoding the I picture or the S picture to be subjected to intra-picture encoding. This is an instruction indicating that all data is deleted from the memory for reference in encoding or decoding.

  Hereinafter, the encoding method will be described with reference to FIG. FIG. 21A shows a procedure for creating a coded signal in the present embodiment.

  First, in Step 01, the picture number PN is detected. Next, in Step 02, the picture number PN detected in Step 01 is encoded. In step 03, picture type information PicType is detected. In Step 03, it is determined whether or not the detected picture type is an S picture.

  If the detected picture type is an S picture, in Step 05, all picture deletion information that means deletion of all the pictures on the memory is encoded. Next, the S picture is encoded in Step 06A. In step 07, the picture number is initialized, and in step 08, all the pictures on the memory other than the S picture to be encoded are deleted. This completes the encoding of the additional information and the initialization process of the picture number PN.

  If the detected picture type is not an S picture, the picture numbers are continuous, so that the picture is encoded in Step 06B, but additional information is encoded, picture number PN is initialized, and all pictures are not deleted. End the process.

  The initialization of the picture number PN in Step 07 is, for example, assigning the picture number 0 to the S picture that has been encoded. That is, by initializing the picture number in the S picture, a number (for example, PN = 1) starting from the S picture (PN = 0) is assigned to the picture that follows the S picture in the order of display time. It means to become. As a result, the picture number PN is initialized after the S picture is encoded (that is, after the picture number of the S picture is encoded).

  In Step 04, it is determined whether or not it is an S picture, but it may be determined whether or not it is an I picture. In FIG. 21A, when there is a step of deleting all the pictures, the picture number may be initialized together, and whether or not the picture number initialization process is an I picture or an S picture is determined. It is not limited. Further, the encoding process of the picture number PN in Step 02 is a process after the process of detecting the picture number in Step 01, and may be performed at any time point before the process of initializing the picture number PN in Step 07. In addition, after the process of deleting all the pictures on the memory other than the S picture to be encoded in Step 08, the picture number PN may be initialized in Step 07. The process of encoding all picture deletion information, which means deleting all the pictures on the memory in Step 05, is a process after determining whether or not the picture is an S picture in Step 04, and is shown in FIG. Any time before the process shown is finished. Also, use special picture type information PicType that includes additional information that means that all information other than the picture to be encoded is deleted from the memory for reference in encoding or decoding is included in the picture type information PicType. Thus, it is possible not to encode the additional information. Furthermore, it is effective to change the picture number PN in order to switch between the S picture and the I picture, but it is not necessarily effective only with the S picture or the I picture. If the stream can be switched, all the P pictures and the like can be changed. If there is a step of deleting a picture, the same processing may be performed on the picture number PN.

  FIG. 22 is a block diagram showing a configuration of an image coding apparatus that implements the coding method according to the twelfth embodiment.

  The picture number generation unit PNGen generates a picture number PN. The picture number PN is an identifier for distinguishing images stored in the reference image memory Mem, and different pictures stored in the reference image memory Mem are assigned different picture numbers PN. Normally, the picture number PN is incremented by “1” every time an image is stored in the reference image memory Mem. Further, according to the notification from the image encoding unit PicEnc, after encoding the S picture, the picture number PN of the S picture is initialized to “0”.

  The image removal unit PicDel5 erases the image stored in the reference image memory Mem other than the picture to be encoded when the picture type information PicType indicates an S picture (corresponding to the process of Step 03 in FIG. 21). Command (all picture deletion information) is notified to the reference image memory Mem, and at the same time, the information is also notified to the variable length coding unit VLC.

  The image coding unit PicEnc refers to the image stored in the reference image memory Mem, encodes the input image signal Vin as a picture type indicated by the picture type information PicType with frequency conversion / quantization, and the result. It transmits to the image decoding unit PicDec and the variable length coding unit VLC. The image encoding unit PicEnc encodes the S picture, and then notifies the picture number generation unit PNGen2 of an instruction to initialize the picture number PN.

  The image decoding unit PicDec performs inverse quantization / inverse frequency conversion on the result of encoding by the image encoding unit PicEnc as the picture type indicated by the picture type information PicType, and the picture number PN for reference in subsequent image encoding And stored in the reference image memory Mem.

  The variable-length encoding unit VLC performs variable-length encoding on the result encoded by the image encoding unit PicEnc to form a bit string, and the reference image memory Mem notified from the image removal unit PicDel5, which is information necessary for decoding The information for erasing the image stored in the image (all picture deletion information), the picture number PN, and the picture type information PicType are encoded and output as an encoded signal Str.

  Next, the decoding method will be described with reference to FIG. FIG. 21B shows a procedure for decoding the encoded signal.

  First, in Step 09, the picture number PN is decoded. Next, in Step 010, it is determined whether or not all picture deletion information has been encoded.

  If it is determined in step 010 that all picture deletion information has been encoded, in step 011 all picture deletion information is decoded. In step 012A, the picture is decoded. Further, all the pictures on the memory other than the picture to be decoded are deleted in Step 013, and then the picture number PN is initialized in Step 014. This completes the decoding of the additional information and the initialization process of the picture number PN.

  If it is determined in Step 010 that all picture deletion information has not been encoded, the picture is decoded in Step 012B, and the decoding of the additional information and the initialization process of the picture number PN are terminated.

  The initialization of the picture number PN in Step 014 is, for example, assigning a picture number “0” to a picture that has been decoded. That is, when decoding the encoded signal encoded by the encoding procedure shown in FIG. 21A, the picture number is initialized in the S picture, thereby succeeding the S picture in the order of display time. This means that a number starting with an S picture is assigned to a picture.

  In FIG. 21B, if there is a step of deleting all the pictures, the process of initializing the picture number may be performed, and whether or not to perform the process is related to the type of picture to be decoded. There is no. Also, the initialization process of the picture number PN in Step 014 may be a process before the process of deleting all the pictures on the memory other than the picture to be encoded in Step 013. In addition, by using special picture type information PicType that includes additional information that means that all information other than the picture to be decoded is deleted from the memory for reference in decoding is included in the picture type information PicType. It is also possible not to encode the information.

  FIG. 23 is a block diagram illustrating a configuration of an image decoding apparatus that implements the decoding method according to the twelfth embodiment.

  The variable length decoding unit VLD decodes the encoded signal Str, and includes various information (a command for deleting an image stored in the reference image memory Mem, picture type information PicType, picture number PN, information for changing the picture number PN, and Image data).

  First, a command (all picture deletion information) for deleting an image stored in the reference image memory Mem obtained by the variable length decoding unit VLD is notified to the image removal unit PicDel6, and the image removal unit PicDel6 Erase the commanded image stored in the memory Mem.

  The picture type information PicType obtained by the variable length decoding unit VLD is notified to the image decoding unit PicDec and indicates the decoding method.

  The picture number PN obtained by the variable-length decoding unit VLD is notified to the reference image memory Mem, and is used as the picture number PN for storing the image decoded by the image decoding unit PicDec.

  All-picture deletion information obtained by the variable-length decoding unit VLD is notified to the picture number change unit PNchg2, and the picture number change unit PNchg2 changes the picture number PN of the image stored in the reference image memory Mem according to the instruction. Change (initialize). More specifically, the picture number changing unit PNchg2 deletes all the pictures other than the decoding target picture (S picture) in the reference picture memory Mem, and then the picture number of the picture stored in the reference picture memory Mem. After the PN is read out and the value of the read picture number PN is changed to “0”, the picture number PN is written into the reference image memory Mem.

  The image data obtained by the variable length decoding unit VLD is decoded by the image decoding unit PicDec by a decoding method according to the picture type indicated by the picture type information PicType. That is, the I picture is decoded without referring to the image in the reference image memory Mem, and the P picture and the B picture are decoded with reference to the image stored in the reference image memory Mem. The decoded image obtained in this way is stored in the reference image memory Mem and is output as a decoded image signal Vout.

  With the above configuration, an image decoding apparatus that realizes the image decoding method shown in FIG. 21 can be realized. Also, it is possible to provide a decoding device with high error tolerance.

  As described above, the memory state of each of the plurality of streams becomes the same after moving from a predetermined stream to another stream by the encoding method and the decoding method shown in the present embodiment. Even when a reference picture is required, a predetermined picture can be accurately designated on the memory.

  In the above embodiment, it has been described that the additional information (all picture deletion information) may be encoded together with the picture PicType. However, in the I picture, the I picture can be deleted by deleting all the pictures in the reference memory. From this, you can make a special picture that can play the stream. This is called an IDR (Instantaneous Decoder Refresh) picture. Since the IDR picture is a random access playback start position, it is effective as the first I picture of a GOP (Group of Picture). Every time when encoding this IDR picture, if it is determined that all the pictures other than the picture in the memory are deleted and the picture number is initialized after the picture is encoded, the image encoding apparatus stores the picture in the memory. Even when all the pictures other than the picture are deleted, the additional information may not be encoded. In this case, in the image decoding apparatus, when the IDR picture in the encoded stream is detected from the picture type and the IDR picture is decoded, even if the additional information is not encoded, each time in the memory All pictures other than the IDR picture are deleted, and the picture number is initialized after the picture is encoded / decoded.

(Embodiment 13)
Furthermore, by recording the program for realizing the configuration of the image encoding method and the image decoding method shown in the above embodiments on a storage medium such as a flexible disk, The illustrated processing can be easily performed in an independent computer system.

  FIG. 24 is an explanatory diagram of a storage medium for storing a program for realizing the image encoding method and the image decoding method of Embodiments 1 to 12 by a computer system.

  FIG. 24B shows the appearance, cross-sectional structure, and flexible disk as seen from the front of the flexible disk, and FIG. 24A shows an example of the physical format of the flexible disk that is the recording medium body. The flexible disk FD is built in the case F, and on the surface of the disk, a plurality of tracks Tr are formed concentrically from the outer periphery toward the inner periphery, and each track is divided into 16 sectors Se in the angular direction. ing. Therefore, in the flexible disk storing the program, the image encoding method and the image decoding method as the program are recorded in an area allocated on the flexible disk FD.

  FIG. 24C shows a configuration for recording and reproducing the program on the flexible disk FD. When the program is recorded on the flexible disk FD, the image encoding method and the image decoding method as the program are written from the computer system Cs via the flexible disk drive. When the image encoding method and the image decoding method are constructed in a computer system by a program in a flexible disk, the program is read from the flexible disk by a flexible disk drive and transferred to the computer system.

  In the above description, a flexible disk is used as the recording medium, but the same can be done using an optical disk. Further, the recording medium is not limited to this, and any recording medium such as a CD-ROM, a memory card, and a ROM cassette that can record a program can be similarly implemented.

  Furthermore, application examples of the image coding method and the image decoding method shown in the above embodiment and a system using the same will be described.

  FIG. 25 is a block diagram showing an overall configuration of a content supply system ex100 that implements a content distribution service. The communication service providing area is divided into desired sizes, and base stations ex107 to ex110, which are fixed radio stations, are installed in each cell.

  The content supply system ex100 includes, for example, a computer ex111, a PDA (personal digital assistant) ex112, a camera ex113, a mobile phone ex114, a camera via the Internet ex101, the Internet service provider ex102, the telephone network ex104, and the base stations ex107 to ex110. Each device such as the attached mobile phone ex115 is connected.

  However, the content supply system ex100 is not limited to the combination shown in FIG. 25, and any combination may be connected. Further, each device may be directly connected to the telephone network ex104 without going through the base stations ex107 to ex110 which are fixed wireless stations.

  The camera ex113 is a device capable of shooting a moving image such as a digital video camera. The mobile phone is a PDC (Personal Digital Communications) system, a CDMA (Code Division Multiple Access) system, a W-CDMA (Wideband-Code Division Multiple Access) system, or a GSM (Global System for Mobile Communications) system mobile phone, Alternatively, PHS (Personal Handyphone System) or the like may be used.

  In addition, the streaming server ex103 is connected from the camera ex113 through the base station ex109 and the telephone network ex104, and live distribution or the like based on the encoded data transmitted by the user using the camera ex113 becomes possible. The encoded processing of the captured data may be performed by the camera ex113 or may be performed by a server or the like that performs data transmission processing. Further, the moving image data shot by the camera ex116 may be transmitted to the streaming server ex103 via the computer ex111. The camera ex116 is a device that can shoot still images and moving images, such as a digital camera. In this case, the encoding of the moving image data may be performed by the camera ex116 or the computer ex111. The encoding process is performed in the LSI ex117 included in the computer ex111 and the camera ex116. Note that image encoding / decoding software may be incorporated into any storage medium (CD-ROM, flexible disk, hard disk, etc.) that is a recording medium readable by the computer ex111 or the like. Furthermore, you may transmit moving image data with the mobile telephone ex115 with a camera. The moving image data at this time is data encoded by the LSI included in the mobile phone ex115.

  In this content supply system ex100, the content (for example, video shot of music live) captured by the user with the camera ex113, camera ex116, etc. is encoded and transmitted to the streaming server ex103 as in the above embodiment. On the other hand, the streaming server ex103 distributes the content data to the requested client. Examples of the client include a computer ex111, a PDA ex112, a camera ex113, a mobile phone ex114, and the like that can decode the encoded data. In this way, the content supply system ex100 can receive and reproduce the encoded data at the client, and also realize personal broadcasting by receiving, decoding, and reproducing in real time at the client. It is a system that becomes possible.

  The image encoding device or the image decoding device described in the above embodiments may be used for encoding and decoding of each device constituting this system.

A mobile phone will be described as an example.
FIG. 26 is a diagram illustrating the mobile phone ex115 that uses the image coding method and the image decoding method described in the above embodiment. The cellular phone ex115 includes an antenna ex201 for transmitting and receiving radio waves to and from the base station ex110, a camera such as a CCD camera, a camera unit ex203 capable of taking a still image, a video shot by the camera unit ex203, and an antenna ex201. A display unit ex202 such as a liquid crystal display that displays data obtained by decoding received video and the like, a main body unit composed of a group of operation keys ex204, an audio output unit ex208 such as a speaker for audio output, and audio input To store encoded data or decoded data such as a voice input unit ex205 such as a microphone, captured video or still image data, received mail data, video data or still image data, etc. Recording medium ex207, and slot portion ex20 for enabling recording medium ex207 to be attached to mobile phone ex115 The has. The recording medium ex207 stores a flash memory element which is a kind of EEPROM (Electrically Erasable and Programmable Read Only Memory) which is a nonvolatile memory that can be electrically rewritten and erased in a plastic case such as an SD card.

  Further, the cellular phone ex115 will be described with reference to FIG. The cellular phone ex115 controls the power supply circuit ex310, the operation input control unit ex304, and the image coding for the main control unit ex311 which is configured to control the respective units of the main body unit including the display unit ex202 and the operation key ex204. Unit ex312, camera interface unit ex303, LCD (Liquid Crystal Display) control unit ex302, image decoding unit ex309, demultiplexing unit ex308, recording / reproducing unit ex307, modulation / demodulation circuit unit ex306, and audio processing unit ex305 via a synchronization bus ex313 Are connected to each other.

  When the end call and power key are turned on by a user operation, the power supply circuit ex310 activates the camera-equipped digital cellular phone ex115 by supplying power from the battery pack to each unit. .

  The cellular phone ex115 converts the audio signal collected by the audio input unit ex205 in the audio call mode into digital audio data by the audio processing unit ex305 based on the control of the main control unit ex311 including a CPU, a ROM, a RAM, and the like. The modulation / demodulation circuit unit ex306 performs spectrum spread processing, and the transmission / reception circuit unit ex301 performs digital analog conversion processing and frequency conversion processing, and then transmits the result via the antenna ex201. The cellular phone ex115 amplifies the received data received by the antenna ex201 in the voice call mode, performs frequency conversion processing and analog-digital conversion processing, performs spectrum despreading processing by the modulation / demodulation circuit unit ex306, and analog audio by the voice processing unit ex305. After the data is converted, it is output via the audio output unit ex208.

  Further, when an e-mail is transmitted in the data communication mode, text data of the e-mail input by operating the operation key ex204 of the main body is sent to the main control unit ex311 via the operation input control unit ex304. The main control unit ex311 performs spread spectrum processing on the text data in the modulation / demodulation circuit unit ex306, performs digital analog conversion processing and frequency conversion processing in the transmission / reception circuit unit ex301, and then transmits the text data to the base station ex110 via the antenna ex201.

  When transmitting image data in the data communication mode, the image data captured by the camera unit ex203 is supplied to the image encoding unit ex312 via the camera interface unit ex303. When image data is not transmitted, the image data captured by the camera unit ex203 can be directly displayed on the display unit ex202 via the camera interface unit ex303 and the LCD control unit ex302.

  The image encoding unit ex312 has a configuration including the image encoding device described in the present invention, and an encoding method using the image data supplied from the camera unit ex203 in the image encoding device described in the above embodiment. The encoded image data is converted into encoded image data by compression encoding, and sent to the demultiplexing unit ex308. At the same time, the cellular phone ex115 sends the sound collected by the audio input unit ex205 during imaging by the camera unit ex203 to the demultiplexing unit ex308 as digital audio data via the audio processing unit ex305.

  The demultiplexing unit ex308 multiplexes the encoded image data supplied from the image encoding unit ex312 and the audio data supplied from the audio processing unit ex305 by a predetermined method, and the multiplexed data obtained as a result is a modulation / demodulation circuit unit A spectrum spread process is performed in ex306, a digital analog conversion process and a frequency conversion process are performed in the transmission / reception circuit unit ex301, and then the signal is transmitted through the antenna ex201.

  When data of a moving image file linked to a homepage or the like is received in the data communication mode, the received data received from the base station ex110 via the antenna ex201 is subjected to spectrum despreading processing by the modulation / demodulation circuit unit ex306, and the resulting multiplexing is obtained. Is sent to the demultiplexing unit ex308.

  In addition, in order to decode multiplexed data received via the antenna ex201, the demultiplexing unit ex308 separates the multiplexed data into a bit stream of image data and a bit stream of audio data, and synchronizes them. The encoded image data is supplied to the image decoding unit ex309 via the bus ex313, and the audio data is supplied to the audio processing unit ex305.

  Next, the image decoding unit ex309 is configured to include the image decoding device described in the present invention, and decodes a bit stream of image data with a decoding method corresponding to the encoding method described in the above embodiment. Thus, the reproduction moving image data is generated and supplied to the display unit ex202 via the LCD control unit ex302, and thereby, for example, moving image data included in the moving image file linked to the home page is displayed. At the same time, the audio processing unit ex305 converts the audio data into analog audio data, and then supplies the analog audio data to the audio output unit ex208. Thus, for example, the audio data included in the moving image file linked to the home page is reproduced. The

  Note that the present invention is not limited to the above-described system, and recently, digital broadcasting by satellite and terrestrial has become a hot topic. As shown in FIG. Any of the decoding devices can be incorporated. Specifically, in the broadcasting station ex409, a bit stream of video information is transmitted to a communication or broadcasting satellite ex410 via radio waves. Receiving this, the broadcasting satellite ex410 transmits a radio wave for broadcasting, and receives the radio wave with a home antenna ex406 having a satellite broadcasting receiving facility, such as a television (receiver) ex401 or a set top box (STB) ex407. The device decodes the bitstream and reproduces it. In addition, the image decoding apparatus described in the above embodiment can also be mounted on a playback apparatus ex403 that reads and decodes a bitstream recorded on a storage medium ex402 such as a CD or a DVD as a recording medium. In this case, the reproduced video signal is displayed on the monitor ex404. Further, a configuration in which an image decoding device is mounted in a set-top box ex407 connected to a cable ex405 for cable television or an antenna ex406 for satellite / terrestrial broadcasting, and this is reproduced on the monitor ex408 of the television is also conceivable. At this time, the image decoding apparatus may be incorporated in the television instead of the set top box. It is also possible to receive a signal from the satellite ex410 or the base station ex107 by the car ex412 having the antenna ex411 and reproduce a moving image on a display device such as the car navigation ex413 that the car ex412 has.

  Further, the image signal can be encoded by the image encoding device shown in the above embodiment and recorded on a recording medium. As a specific example, there is a recorder ex420 such as a DVD recorder that records an image signal on a DVD disk ex421 or a disk recorder that records on a hard disk. Further, it can be recorded on the SD card ex422. If the recorder ex420 includes the image decoding device described in the above embodiment, the image signal recorded on the DVD disc ex421 or the SD card ex422 can be reproduced and displayed on the monitor ex408.

  For example, the configuration of the car navigation ex413 may be a configuration excluding the camera unit ex203, the camera interface unit ex303, and the image encoding unit ex312 in the configuration illustrated in FIG. 27. The same applies to the computer ex111 and the television (receiver). ) Ex401 can also be considered.

  In addition to the transmission / reception type terminal having both the encoder and the decoder, the terminal such as the mobile phone ex114 has three mounting formats: a transmitting terminal having only an encoder and a receiving terminal having only a decoder. Can be considered.

  As described above, the moving picture encoding method or the moving picture decoding method described in the above embodiment can be used in any of the above-described devices and systems, and as a result, the above-described embodiment has been described. An effect can be obtained.

  In addition, the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the present invention.

  The image encoding apparatus according to the present invention is useful as an image encoding apparatus provided in a personal computer, a PDA, a digital broadcast station, a mobile phone, and the like having a communication function.

  The image decoding apparatus according to the present invention is useful as an image decoding apparatus provided in a personal computer having a communication function, a PDA, an STB that receives digital broadcasting, a mobile phone, and the like.

It is explanatory drawing of the picture number PN of the image stored in the reference image memory Mem. It is a flowchart of the encoding method and the decoding method of the information regarding the image control stored in the reference image memory Mem of the image encoding method and image decoding method of this invention. It is explanatory drawing of the picture number PN of the image stored in the reference image memory Mem. It is a flowchart of the encoding method and the decoding method of the information regarding the image control stored in the reference image memory Mem of the image encoding method and image decoding method of this invention. FIG. 6 is a diagram illustrating correspondence between a picture and a picture number PN when an input image signal Vin of the present invention is encoded. It is a flowchart of the decoding method of the information regarding the image control stored in the reference image memory Mem of the image decoding method of this invention. It is a block diagram which shows the structure of the image coding apparatus of this invention. It is a figure which shows an example of the data structure of the encoding signal Str of this invention. It is a block diagram which shows the structure of the image decoding apparatus of this invention. FIG. 6 is a diagram illustrating correspondence between each picture and a picture number PN when an input image signal Vin of the present invention is encoded. It is a flowchart which shows the method of assign | providing and encoding a picture number to each picture of the stream of this invention. FIG. 38 is a flowchart illustrating a decoding method according to the seventh embodiment. It is a figure which shows the memory structure in this Embodiment 7. It is a flowchart which shows the encoding method in this Embodiment 7. It is a flowchart which shows another encoding method in this Embodiment 7. It is a flowchart which shows another encoding method in this Embodiment 7. FIG. 29 is a block diagram showing a configuration of an encoding apparatus according to the eighth embodiment. FIG. 38 is a block diagram showing a configuration of another encoding apparatus according to the eighth embodiment. FIG. 38 is a flowchart showing a decoding method in the tenth embodiment. It is a block diagram which shows the structure of the decoding apparatus in this Embodiment 11. FIG. FIG. 38 is a flowchart illustrating a coded signal creation procedure and a coded signal decoding procedure in the twelfth embodiment. It is a block diagram which shows the structure of the image coding apparatus which implement | achieves the encoding method of this Embodiment 12. FIG. It is a block diagram which shows the structure of the image decoding apparatus which implement | achieves the decoding method of this Embodiment 12. FIG. FIG. 38 is an explanatory diagram of a storage medium that stores a program for realizing the image coding method and the image decoding method according to the first to twelfth embodiments by a computer system. It is a block diagram which shows the whole structure of the content supply system which implement | achieves the content delivery service which concerns on this invention. It is a figure which shows an example of the mobile telephone which concerns on this invention. It is a block diagram which shows the structure of the mobile phone. It is a figure which shows the structure of the system for digital broadcasting which concerns on this invention. It is a figure explaining a response | compatibility with the picture at the time of encoding the input image signal Vin and picture number PN. It is a figure which shows the picture number PN of the image stored in the reference image memory Mem when encoding and decoding an S picture.

PicType Picture type information
Vin input image signal
PNcmp Memory number comparison unit
Mem memory
PicEnc image encoding unit
PNGen picture number generation unit
VLC variable length coding unit
PicDec image decoding unit
PicDel4 image removal unit

Claims (12)

An image encoding method for encoding a moving image,
Assign a picture number to the picture to be encoded,
A picture number of the picture to be encoded is encoded;
Determining whether all reference picture deletion information, which is instruction information for deleting all reference pictures stored in the memory, is included in the encoding target picture;
If it is determined that all reference picture deletion information is included in the encoding target picture,
Encoding the all reference picture deletion information;
Encoding the encoding target picture and outputting the encoded picture;
Decoding the encoded picture to generate a decoded reference picture;
After decoding the picture to be encoded, delete all reference pictures stored in the memory,
Storing the decoded reference picture in a memory;
After decoding the picture to be coded, the picture number of the picture to be coded is initialized, and a picture number subsequent to the initialized picture number is coded to follow the picture to be coded To the picture,
Encoding the subsequent picture number;
An image encoding method, wherein the subsequent picture is encoded. The picture stored in the memory is a reference picture, and all the pictures deleted in the deletion process of all the pictures stored in the memory are also reference pictures. Image coding method. The deletion process of all the pictures stored in the memory is executed by setting information indicating deletion for all the decoded pictures stored in the memory. 2. The image encoding method according to 1. The image coding method according to claim 1, wherein the all reference picture deletion information is information indicating an IDR picture. The image encoding method according to claim 4, wherein the IDR picture is an intra prediction encoding picture. The image coding method according to claim 1, wherein the all reference picture deletion information is picture type information indicating that the picture is an intra-picture prediction coded picture. An image encoding device for encoding a moving image,
Picture number assigning means for assigning a picture number to a picture to be encoded;
Picture number encoding means for encoding a picture number of the picture to be encoded;
It is determined whether or not all reference picture deletion information, which is instruction information for deleting all reference pictures stored in the memory, is included in the encoding target picture, and is included in the encoding target picture When it is determined that all reference picture deletion information is included, deletion information encoding means for encoding the all reference picture deletion information,
Picture encoding means for encoding the encoding target picture and outputting the encoded picture;
Picture decoding means for decoding the encoded picture to generate a decoded reference picture;
Memory management means for deleting all reference pictures stored in the memory after decoding the encoding target picture;
Storage means for storing the decoded reference picture in a memory;
After decoding the encoding target picture, the picture number of the encoding target picture is initialized, and a picture number subsequent to the initialized picture number is encoded following the encoding target picture. An initialization means for assigning to the picture,
The picture number encoding means encodes the subsequent picture number;
The picture encoding device encodes the subsequent picture. The image encoding apparatus, wherein: 8. The image coding according to claim 7, wherein the picture stored in the memory by the storage means is a reference picture, and pictures that are all disabled by the memory management means are also reference pictures. apparatus. 8. The memory management unit deletes all the pictures stored in the memory by setting information indicating that all the pictures stored in the memory are deleted. The image encoding device described in 1. The image coding apparatus according to claim 7, wherein the all reference picture deletion information is information indicating an IDR picture. The image coding apparatus according to claim 10, wherein the IDR picture is an intra-picture prediction coding picture. The image coding apparatus according to claim 7, wherein the all reference picture deletion information is picture type information indicating that the picture is an intra-frame prediction coded picture.

Images