JP2008047967A - Reproducing unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To cope with the decoding of image data encoded by other pieces of equipment having a reference relationship not less than own memory capacity. <P>SOLUTION: A CPU 18 checks whether there is a reference picture required for the decoding in the reference picture region of a picture memory 16. When there are no reference pictures required for decoding, the decoding is restarted after returning to an IDR (Instantaneous Decoder Refresh) picture. In the restarted decoding, a resizing device 22 reduces the picture size of the image data decoded by a codec 12 for storing in the picture memory 16. An up-converter 24 resizes a picture read from the picture memory 16 for display to its original size for applying to a display device 20. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video decoding apparatus that decodes compressed video data compressed using intra-picture coding and inter-picture prediction coding.

  In recent years, many apparatuses for recording / reproducing digital data such as audio or video have been developed by improving the processing speed of the information processing apparatus. In particular, recently, as memory cards and disk media used for external storage media have been increasing in capacity and price, devices with functions for recording and playing back moving images and audio on these recording media have increased. ing.

  For example, digital video cameras having a function of recording / reproducing moving images not only on a tape medium but also on a memory card or a disk medium are widely available, and video cameras that do not use a tape medium have also appeared. When video or audio is recorded with such a device, the data amount is usually reduced by compressing the data. MPEG1 and MPEG2 defined by the Moving Picture Experts Group (MPEG) are commonly used video compression / encoding standards. MPEG-1 compresses a moving picture signal to 1.5 Mbps, and MPEG-2 compresses a moving image signal to 2 to 15 Mbps.

  Currently, in order to meet the demand for higher image quality, H.264 is the next generation screen coding system jointly developed by ISO / IEC and ITU. H.264 / MPEG-4 Part 10: AVC (hereinafter simply referred to as H.264) was standardized. In general, in video encoding, the amount of data is compressed by reducing redundancy in the time direction and the spatial direction. Therefore, in inter-picture predictive coding for the purpose of reducing temporal redundancy, motion is detected and a predicted image is created in units of blocks with reference to forward or backward pictures, and the obtained predicted image and the current The difference value with the picture is encoded.

  A picture type at the time of decoding in predictive coding and a reference relationship thereof will be described with reference to FIGS. 13 and 14. FIG. A reference relationship example in the case of H.264 is shown, and FIG. 14 shows a reference relationship example in the case of MPEG2.

  An image (frame or field, or part thereof) that can be decoded without a reference picture, such as picture I0, is called an I picture. A picture that is decoded with reference to only one picture, such as picture P3, is called a P picture. A picture that is coded by referring to two pictures at the same time is a B picture.

  In the MPEG2 example shown in FIG. 14, the P picture refers only to the immediately preceding I picture or P picture. The B picture refers only to the I picture or P picture immediately before and after. Therefore, the reference pictures necessary for decoding are a maximum of two pictures that can be discriminated for each picture type of the picture to be decoded.

  On the other hand, H. In the case of H.264, each picture can be referred from any picture within a predetermined period as in the example shown in FIG.

  In this way, H.C. In H.264, encoding efficiency is improved by adopting flexible inter-screen prediction. On the other hand, in order to realize this, H.C. In H.264, compared to MPEG2, more reference pictures must be stored in a reference picture memory for temporarily storing decoded pictures, and more in order to rearrange them in display order, Memory capacity and computation are required.

Japanese Patent Application Laid-Open No. 2004-228867 describes a configuration that reduces the number of images to be stored in the reference picture memory by controlling the reference relationship.
JP-A-2005-260588

  However, in a device for which the memory capacity is to be reduced as much as possible, for example, a consumer-use portable video camera, adding a reference picture memory leads to an increase in cost and enlargement of the apparatus, so it is desired to suppress it as much as possible.

  Also, from the viewpoint of consumption of a battery serving as a drive source of the video camera, it is better to suppress the memory capacity as much as possible and to suppress the amount of data stored therein. Furthermore, it is considered that the technique described in Patent Document 1 is not suitable for a consumer video camera used for photographing a subject that moves rapidly such as an athletic meet.

  Therefore, if a necessary reference picture can be stored without using as much memory capacity as possible, it is very useful for a video camera.

  On the other hand, if the memory capacity is simply designed to be small, a self-recording stream can be decoded without any problem, but for example H.264 having a complicated reference relationship generated by another device. When an H.264 stream is input, a situation may occur in which decoding of the reference picture required for the reference picture storage memory can be performed and decoding is broken. That is, H.264 having a complicated or long-time reference relationship encoded by another device. Considering the situation of decoding a H.264 stream, decoding simply fails when the number of images that can be stored is simply reduced.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a video decoding apparatus that is unlikely to cause a decoding failure even when the memory capacity of a reference picture storage memory is small. .

  In order to achieve the above object, a playback device according to the present invention refers to a decoding means for decoding encoded data with reference to a reference picture, a picture memory for temporarily storing the reference picture, and the decoding A reference picture determining means for determining whether or not a reference picture necessary for decoding by the means is stored in the picture memory, and reducing the picture size of the picture decoded by the decoding means to the picture memory A resize means for writing and an up-converter for returning a picture read from the picture memory for display to the original picture size, and a reference picture required for decoding by the decoding means is stored in the picture memory Includes a picture that is to be a reference picture decoded by the decoding means. When the picture that has been written and read from the picture memory for display is output to the outside and the reference picture required for decoding by the decoding means is not stored in the picture memory, the picture is returned to the predetermined picture. The picture to be the reference picture decoded by the decoding means is written to the picture memory via the resizing means, and the picture read out for display from the picture memory is sent via the up-converter. Output to the outside.

  A playback apparatus according to the present invention includes a decoding unit that decodes encoded data with reference to a reference picture, a picture memory that temporarily stores the reference picture, and a reference picture necessary for decoding by the decoding unit A reference picture determination means for determining whether or not is stored in the picture memory, and a field selection means for selecting one field of the picture decoded by the decoding means and writing to the picture memory, When a reference picture necessary for decoding by the decoding unit is stored in the picture memory, a picture to be a reference picture decoded by the decoding unit is written in the picture memory, and the decoding is performed. If the reference picture required for decoding by the conversion means is not stored in the picture memory, it returns to the predetermined picture , Decoded by the decoding means, said one field of the picture to be a reference picture, characterized in that it is written in the picture memory by the field selection means.

  According to the present invention, the number of reference pictures that can be held in a reference picture area that stores reference pictures can be increased, so that it is possible to prevent a failure of decoding due to the inability to hold a reference memory.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic block diagram of an embodiment of the present invention. The playback device 10 includes a codec 12, a recording medium 14, a picture memory 16, a CPU 18, a display device 20, a resizing device 22, and an upconverter 24. The codec 12 is H.264. From an encoder (encoding device) that compresses and encodes a digital image signal using the H.264 image compression method and generates compressed image data, and a decoder (decoding device) that decodes the compressed image data and restores the original image data Become.

  The recording medium 14 is composed of a non-volatile semiconductor memory, a magnetic disk such as an HDD (Hard Disk Drive), or a recordable / reproducible optical disk conforming to a DVD (Digital Versatile Disk).

  The picture memory 16 is a memory that temporarily stores decoded image data (pictures), that is, a so-called decoded picture buffer DPB (Decoded Picture Buffer). As shown in FIG. The reference picture area for storing the image data and the non-reference picture area for storing the image data of the picture that is not referred to in the decoding. The CPU 18 controls the entire playback device 10. The display device 20 is composed of, for example, an LCD (liquid crystal display panel).

  The resizing device 22 is a device that reduces the size of a picture or thins out data or pixels. On the contrary, the up-converter 24 supplements the size, data, or pixels to return to the original size, data number, or pixel number. Device. Although details will be described later, in this embodiment, when the number of reference pictures exceeds the capacity of the reference memory area of the picture memory 16, the size of each picture is reduced, that is, the data amount is reduced by half by thinning out pixels. Thereby, the capacity of the picture memory 16 is substantially doubled. As it is, since the number of pixels is small and display is hindered, the up-converter 24 interpolates the pixels of the display picture read from the picture memory 16 for display output.

  First, a basic management method of the picture memory 16 will be described. H. Since H.264 allows a very complicated reference relationship, a method for managing a plurality of decoded pictures with the DPB is defined. First, the decoded image data is designated for reference / non-reference in units of pictures. Since the number of reference pictures that can be stored in the picture memory 16 itself or in its reference picture area is limited, in order to store a new reference picture in the picture memory 16, the existing reference picture must be deleted from the picture memory 16. Don't be. As a method, when a certain number of referable pictures in the picture memory has reached a certain number, a moving window (Sliding Window) memory management method for moving or deleting the oldest picture to a non-reference picture area and a management command are used. An adaptive memory management method (MMCO) is known in which an arbitrary picture in a reference picture area of the picture memory 16 is moved or deleted to a non-reference picture area.

  FIG. 3 shows a management flow of the picture memory 16 by the moving window memory management method. The codec 12 decodes a certain picture (S10). It is determined whether or not there is an empty area in the reference picture area of the picture memory 16 (S11).

  If there is no empty area in the reference picture area (S11), it is determined whether or not there is an empty area in the non-reference picture area of the picture memory 16 (S12). If there is no empty area in the non-reference picture area (S12), the earliest non-reference picture is deleted (S13), and the oldest reference picture in the reference picture area is moved to the non-reference picture area (S14). If there is an empty area in the non-reference picture area (S12), the oldest reference picture in the reference picture area is moved to the non-reference picture area (S14). Since the oldest reference picture is moved to the non-reference picture area, the reference picture area is vacant, so the decoded picture is stored in the reference picture area (S15).

  If there is a vacant space from the reference picture area (S11), the decoded picture is stored in the reference picture area (S15).

  FIG. 4 shows a management flow of the picture memory 16 by the adaptive memory management method. The codec 12 decodes a certain picture (S20). It is determined whether or not the empty area is in the non-reference picture area of the picture memory 16 (S21).

  When there is no free space in the non-reference picture area (S21), the non-reference picture that is displayed earliest is deleted from the non-reference picture area (S22), and the CPU 20 analyzes the ID of the management command, An arbitrary reference picture in the reference picture area is released, and when the decoded picture becomes a reference picture, it is stored in the reference picture area of the picture memory 16 (S23). FIG. 15 shows a correspondence table between management commands and their operations. In step S23, the management command ID is analyzed, and the reference picture is stored and released according to the ID. In the table shown in FIG. 15, both the short-time reference picture and the long-time reference picture are H.264. The short-time reference picture is a normal reference picture that is held for a predetermined period in the picture memory 16 that functions like a FIFO. On the other hand, the long-term reference picture is a long-term reference picture whose setting is changed so as to keep the short-term reference picture in the picture memory 16 and can be used as a reference picture until it is released.

  When there is a free space in the non-reference picture area (S21), the CPU 20 analyzes the management command, releases an arbitrary reference picture in the reference picture area of the picture memory 16, and the decoded picture becomes a reference picture Is stored in the reference picture area of the picture memory 16 (S23).

  The management of the picture memory 16 by the method shown in FIGS. 3 and 4 is based on the premise that the capacity of the reference picture area of the picture memory 16 is sufficiently large with respect to the number of reference pictures. However, when a video that requires a reference picture number exceeding the capacity of the reference picture area of the picture memory 16 is decoded, the decoding fails. Such a situation can occur, for example, when a video stream encoded by another encoding device is decoded.

  In this embodiment, even when the reference picture necessary for decoding cannot be stored in the picture memory 16 even by management of the picture memory 16 shown in FIG. 3 or FIG. 4, the CPU 18 starts the IDR (Instantaneous) from the video stream to be decoded. Decoding Refresh) The video stream to be decoded is redrawn until the picture is reached, and decoding is performed again from the IDR picture. That is, as shown in FIG. 5, the CPU 16 determines whether or not a reference picture required for decoding is in the picture memory 16 (S50). If there is no reference picture (S50), the IDR picture is redrawn from the video stream to be decoded (S51). H. This is because in the H.264 standard, it is prohibited to refer to a past picture beyond the IDR picture, and therefore it is guaranteed that the compressed image can be decoded if decoding is started from the IDR picture.

  After redrawing the video stream, the CPU 18 supplies the decoded image data from the codec 12 to the picture memory 16 via the resizing device 22. That is, as in steps S16 and S24 of the flow shown in FIGS. 6 and 7, the size or the number of pixels of the decoded picture is reduced or thinned by the resizing device 22 and stored in the picture memory 16. FIG. 6 shows a management flow of the picture memory 16 by the moving window memory management method when the resizing device 22 is used, and FIG. 7 shows the management flow of the picture memory 16 by the adaptive memory management method when the resizing device 22 is used. The management flow is shown. The subsequent processing is the same as that in FIGS. 3 and 4 and will not be described.

  The image data read out for display from the picture memory 16 is returned to the original number of pixels by the up-converter 24 and then supplied to the display device 20. FIG. 8 shows a flowchart of the display operation. The image data of each picture read from the picture memory 16 is applied to the upconverter 24, and the upconverter 24 returns the size of the input picture to the original size (S30). The output image data of the up converter 24 is applied to the display device 20, and the display device 20 displays an image indicated by the image data from the up converter 24 (S31).

  The resizing device 22 reduces the size of the decoded image data to 1/2 for the first redrawing, and the up-converter 24 returns the size of each picture from the picture memory 16 to the original size. Even if the resizing device 22 reduces the picture size to ½, if the capacity of the reference picture area is insufficient in the picture memory 16, the CPU 18 reduces the resizing rate in the resizing device 22 to ¼. Also at this time, the CPU 16 changes the conversion rate of the upconverter 24 so that the upconverter 24 returns the size of each picture from the picture memory 16 to the original size.

  FIG. 9 shows an image of the transition of the number of pictures that can be stored in the picture memory 16. FIG. 9A shows the number of pictures that can be initially stored in the picture memory 16. FIG. 9B shows the number of pictures that can be stored in the picture memory 16 when the resizing rate by the resizing device 22 is ½. FIG. 9C shows the number of pictures that can be stored in the picture memory 16 when the resizing rate by the resizing device 22 is 1/4.

  FIG. 10 shows a schematic block diagram of the second embodiment of the present invention. The same components as those in the embodiment shown in FIG. In this embodiment, as a reference picture, only one field of the interlace signal is stored in the picture memory 16 as a reference picture. Thereby, a substantially double number of pictures can be stored in the reference picture area of the picture memory 16. For this purpose, a field selection device 26 for selecting a field to be stored in the picture memory 16 is added to the playback device 10a.

  The CPU 18a sets a field designated in advance in the field selection device 26. The field selection device 26 stores the image data decoded by the codec 12 in the non-reference picture area of the picture memory 16 for display, and among the decoded image data, the picture of the field picture used for reference. Data is stored in the reference picture area of the picture memory 16.

  Also in the present embodiment, when the reference picture required for decoding cannot be stored in the picture memory 16 even by management of the picture memory 16 shown in FIG. 3 or 4 according to FIG. The image data of a specific field is stored in the reference picture area of the picture memory 16 as a reference picture.

  FIG. 11 shows a management flow of the picture memory 16 by the moving window memory management method when the field selection device 26 is used. The field selection device 26 selects a picture of a field used for reference from the image data decoded by the codec 12 (S16a), and stores the image data of the field in the reference picture area of the picture memory 16 (S15a). . The operation of the other parts is the same as that in FIG.

  FIG. 12 shows a management flow of the picture memory 16 by the adaptive memory management method when the field selection device 26 is used. Instead of step S24 of FIG. 7, the field selection device 26 selects a picture of a field used for reference from the image data decoded by the codec 12 (S24a). Then, instead of step S23 in FIG. 7, the CPU 18a deletes an unnecessary reference picture from the reference picture area or moves it to the non-reference picture area based on the analysis result of the management command, and the field selection device 26 selects the field. The image data is stored in the reference picture area of the picture memory 16 (S23a). The operation of the other parts is the same as that in FIG.

  In the third embodiment, since only one field of the reference picture is stored in the picture memory 16, the reference picture area of the picture memory 16 can substantially accommodate twice as many pictures.

It is a schematic block diagram of Example 1 of the present invention. 3 is a schematic diagram of a reference picture area and a non-reference picture area of the picture memory 16. FIG. It is the management flowchart of the picture memory by a moving window memory management method. It is a management flowchart of the picture memory by the adaptive memory management method. It is a flowchart of IDR picture drawing operation. It is a management flowchart of the picture memory by the moving window memory management method after IDR picture drawing. It is the management flowchart of the picture memory by the adaptive memory management method after IDR picture drawing. It is a process flowchart of the picture for a display by an up converter. It is a schematic diagram which shows the transition of the number of pictures which can be accommodated in the picture memory by a present Example. It is a schematic block diagram of Example 2 of this invention. FIG. 11 is a picture memory management flowchart according to the moving window memory management method after the IDR picture is pulled in in Embodiment 2. FIG. 10 is a picture memory management flowchart according to the adaptive memory management method after IDR picture pull-in according to the second embodiment. H. 2 is a diagram illustrating a reference relationship of H.264. It is a figure which shows the reference relationship of MPEG2. It is a correspondence table between IDs and operations of management commands referred to in step S23.

Explanation of symbols

10, 10a Playback device 12 Codec 14 Recording medium 16 Picture memory 18 CPU
20 Display device 22 Resizing device 24 Upconverter 26 Field selection device

Claims (4)

Decoding means (12) for decoding encoded data with reference to a reference picture;
A picture memory (16) for temporarily storing the reference picture;
Reference picture determination means (18) for determining whether or not a reference picture required for decoding by the decoding means is stored in the picture memory;
A resizing means (22) for reducing the picture size of the picture decoded by the decoding means and writing it in the picture memory;
Up-converter (24) for returning a picture read from the picture memory for display to the original picture size
And
When a reference picture necessary for decoding by the decoding unit is stored in the picture memory, a picture to be a reference picture decoded by the decoding unit is written in the picture memory, and the picture The picture read out from the memory for display is output to the outside,
If the reference picture required for decoding by the decoding means is not stored in the picture memory, the picture to be the reference picture decoded by the decoding means is returned to the predetermined picture, A reproduction apparatus characterized in that a picture written to the picture memory via the means and read out for display from the picture memory is output to the outside via the up-converter. Decoding means (12) for decoding encoded data with reference to a reference picture;
A picture memory (16) for temporarily storing the reference picture;
Reference picture determination means (18) for determining whether or not a reference picture required for decoding by the decoding means is stored in the picture memory;
Field selection means (26) for selecting one field of the picture decoded by the decoding means and writing it in the picture memory
And
When a reference picture necessary for decoding by the decoding unit is stored in the picture memory, a picture to be a reference picture decoded by the decoding unit is written in the picture memory,
When the reference picture required for decoding by the decoding means is not stored in the picture memory, the one picture field that is to be the reference picture decoded by the decoding means is returned to the predetermined picture. A reproducing apparatus written to the picture memory by the field selecting means.   When the reference picture determination unit detects that a reference picture necessary for decoding does not exist in the picture memory, the detection of the predetermined picture that can be decoded without the reference picture is decoded. 3. The playback apparatus according to claim 1, wherein the output display is frozen so as to continue displaying the picture displayed immediately before.   The predetermined picture is H.264. The playback apparatus according to claim 1, wherein the playback apparatus is an H.264 IDR (Instantaneous Decoder Refresh) picture.

Images