JP2006270683A - Coding device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem of a conventional technology, wherein a bus bandwidth is pressed, resulting in deteriorating the processing performance because data are frequently transferred from a frame memory, at motion detection. <P>SOLUTION: A motion compensation section 60 includes an advance read purpose SRAM 64, in addition to a SRAM 66 in order to store pixel data transferred from the frame memory 80. A motion vector detection section 62 transfers, in advance, the data of pixel regions with a high frequency of reference to the advance read purpose SRAM 64, within a prescribed searching range of a reference image stored in the frame memory 80. The motion vector detection section 62 transfers pixel data from the frame memory 80 to the SRAM 66, whenever the motion vector detection section 62 searches the pixel data required to be referenced in the case of searching the motion vector. The motion vector detection section 62 refers to the data, by switching between the advance read purpose SRAM 64 and the SRAM 66 to advance motion search. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an encoding apparatus and method for encoding a moving image.

  Broadband networks are rapidly developing, and there are high expectations for services that use high-quality moving images. In addition, a large-capacity recording medium such as a DVD is used, and a user group who enjoys high-quality images is expanding. There is compression coding as an indispensable technique for transmitting moving images via a communication line or storing them in a recording medium. As an international standard for moving image compression coding technology, the MPEG4 standard and H.264 standard. There is a H.264 / AVC standard. Further, there is a next-generation image compression technology such as SVC (Scalable Video Codec) having a high-quality stream and a low-quality stream in one stream.

In compression encoding and decoding of a moving image, moving image frames are stored in a frame memory, and motion compensation is performed with reference to the frame memory. Therefore, data transfer from the frame memory is frequently performed. In particular, in order to generate a higher quality moving image, motion search may be performed in units of decimal pixels, the amount of data handled by motion compensation is increasing, and the memory bandwidth tends to be a processing bottleneck. Patent Document 1 discloses a digital image decoding apparatus that can improve the bandwidth use efficiency of a frame memory.
JP 11-298903 A

  In order to detect motion of the target macroblock of the encoding target frame at the time of compression encoding of the moving image, a pixel area within a predetermined search range of the reference frame is read from the frame memory, and the target macroblock is detected within the pixel area. Search for a macroblock that matches. In motion detection, since the search is repeated to find a macroblock of a reference frame that matches the target macroblock, the number of reads from the frame memory increases, the amount of data transfer increases, and the transfer bandwidth of the frame memory is reduced. Access to the frame memory becomes a bottleneck, which causes a problem that the processing speed of compression encoding decreases.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a moving picture coding technique capable of reducing the access to the frame memory and increasing the coding processing efficiency.

  In order to solve the above-described problem, an encoding apparatus according to an aspect of the present invention is an encoding apparatus that encodes a frame of a moving image, and is referred to when motion detection of a target block of an encoding target frame is performed. A frame memory that holds a reference frame; and a motion detection unit that refers to the reference frame held in the frame memory and repeats motion search to detect a motion of the target block. The motion detection unit has a prefetch memory that transfers pixel data frequently referenced during motion search from the frame memory and holds the pixel data in the reference frame.

  “Pixel data that is frequently referred to” is, for example, pixel data in an area that may be repeatedly referred to a predetermined number of times or more when motion search is repeatedly performed.

  According to this aspect, the amount of data transferred from the frame memory can be reduced.

  Another aspect of the present invention is an encoding method. This method has a high reference frequency in the pixel data of the motion search range in the reference frame from the frame memory that holds the reference frame to be referred to when the motion of the target block of the encoding target frame of the moving image is detected. If the pixel data is transferred in advance and stored in the pre-read memory, and the pixel data to be referred to in motion search is in the pre-read memory, the pixel data stored in the pre-read memory is referred to. Performs motion search by referring to pixel data held in the frame memory.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to improve the processing efficiency of moving picture encoding.

  FIG. 1 is a configuration diagram of an encoding apparatus 100 according to an embodiment. These configurations can be realized in hardware by a CPU, memory, or other LSI of an arbitrary computer, and in software, it is realized by a program having an image encoding function loaded in the memory. Here, functional blocks realized by the cooperation are depicted. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The encoding apparatus 100 according to the present embodiment includes an MPEG series standard that is an ISO / IEC standard and an H.264 standard that is an ITU-T standard. H.26x series standards, or H.264, the latest video compression coding standard developed by both standardization groups. In accordance with the H.264 / AVC standard, a moving image is encoded.

  In the MPEG series standard, an image frame for intra-frame encoding is an I (Intra) frame, an image frame for forward inter-frame predictive encoding with a past frame as a reference image, a P (Predictive) frame, and past and future An image frame that performs bidirectional inter-frame predictive coding using this frame as a reference image is referred to as a B frame.

  On the other hand, H. In H.264 / AVC, a frame that can be used as a reference image may be a past two frames as a reference image or a future two frames as a reference image regardless of the time. Further, three or more frames can be used as the reference image regardless of the number of frames that can be used as the reference image. Therefore, in MPEG-1 / 2/4, the B frame refers to a Bi-directional prediction frame. Note that in H.264 / AVC, the B frame refers to a bi-predictive prediction frame because the time of the reference image does not matter before and after.

  In the specification of the present application, a frame and a picture are used in the same meaning, and an I frame, a P frame, and a B frame are also called an I picture, a P picture, and a B picture, respectively.

  The encoding apparatus 100 receives an input of a moving image in units of frames, encodes the moving image, and outputs an encoded stream. The input moving image frame is stored in the frame memory 80.

  The motion compensation unit 60 uses a past or future image frame stored in the frame memory 80 as a reference image, performs motion compensation for each macroblock of the P frame or the B frame, and generates a motion vector and a predicted image. . The motion compensation unit 60 takes the difference between the P frame or B frame image to be encoded and the predicted image, and supplies the difference image to the DCT unit 20. In addition, the motion compensation unit 60 supplies the generated motion vector to the variable length coding unit 90.

  The DCT unit 20 performs discrete cosine transform (DCT) on the image supplied from the motion compensation unit 60, and gives the obtained DCT coefficient to the quantization unit 30.

  The quantization unit 30 quantizes the DCT coefficient and provides it to the variable length coding unit 90. The variable length coding unit 90 performs variable length coding on the quantized DCT coefficient of the difference image together with the motion vector supplied from the motion compensation unit 60, and generates an encoded stream. The variable length encoding unit 90 performs processing of rearranging the encoded frames in time order when generating the encoded stream.

  In the case of P frame or B frame encoding processing, the motion compensation unit 60 operates as described above. However, in the case of I frame encoding processing, the motion compensation unit 60 does not operate and is not shown here. The I frame is supplied to the DCT unit 20 after intra prediction.

  The motion compensation unit 60 according to the present embodiment performs prefetching of the motion vector search range. First, a general configuration of the motion compensation unit 60 will be described for comparison, and then a configuration of the motion compensation unit 60 that performs prefetching will be described.

  FIG. 2 is a diagram illustrating a general configuration of the motion compensation unit 60. The frame memory 80 and the motion compensation unit 60 are connected by an SBUS 82. The motion compensation unit 60 designates an address, requests data from the frame memory 80, and receives data transmitted from the frame memory 80 via the SBUS 82.

  The motion compensation unit 60 includes an SRAM 66, a motion vector detection unit 62, and a motion compensation prediction unit 68. The motion vector detection unit 62 transfers pixel data within a predetermined search range of the reference image held in the frame memory 80 to the SRAM 66.

  The frame memory 80 is constituted by a large capacity SDRAM as an example, and is accessed via the SBUS 82. On the other hand, the SRAM 66 is formed in the same integrated circuit as the motion vector detection unit 62 and can be accessed from the motion vector detection unit 62 at high speed. The SRAM 66 has a limited capacity compared to the frame memory 80 and functions as a high-speed cache memory for the frame memory 80. Since the SRAM 66 has high data transfer speed, it is suitable for the motion vector detection unit 62 to frequently refer to the pixel area in the SRAM 66 and perform motion search. In order to increase the number of read ports, a plurality of SRAMs 66 are generally provided.

  The motion vector detection unit 62 refers to the pixel data transferred to the SRAM 66 and detects a motion vector. The motion vector detection unit 62 detects a predicted macroblock having the smallest error with respect to the target macroblock from the reference image, and obtains a motion vector indicating the motion from the target macroblock to the predicted macroblock. Since motion detection is performed by searching for a reference macroblock in a reference image that matches a target macroblock in units of integer pixels or decimal pixels, the search is usually repeated a plurality of times in the pixel area. The reference macroblock that best matches the target macroblock is selected as the predicted macroblock.

  The motion compensation prediction unit 68 performs motion compensation on the target macroblock using the motion vector, generates a prediction image, and outputs a difference image between the encoding target image and the prediction image to the DCT unit 20.

  When a tracking method or gradient method in which a search is repeatedly performed while obtaining a search direction is used as a motion vector search method, pixel data in a range referred to in each search is transferred from the frame memory 80 to the SRAM 66 for each search. The For example, it is assumed that a search for a macroblock of 16 pixels in length and width is repeated six times to find a predicted macroblock that matches the target macroblock. In this case, in one search, a pixel area of 21 pixels vertically and horizontally including pixels around the macroblock is transferred to the SRAM 66. If this is repeated six times, the data transfer amount required to search for a motion vector for one target macroblock is 21 × 21 × 6 = 2646 bytes. However, the information amount of one pixel is set to 1 byte for simplicity. In the tracking method and the gradient method, the data transfer amount varies depending on the number of searches.

  As another motion vector search method, the search range is determined and all macroblocks are picked up one by one, matching is calculated with the target macroblock, and the best matching macroblock is predicted macroblock. There is a total search method to find as When the full search method is used, all pixel data within a predetermined search range is transferred from the frame memory 80 to the SRAM 66. In this case, the motion vector detection unit 62 searches for a motion vector in the pixel area transferred to the SRAM 66. In the full search method, data is transferred only once. For example, when the search range is ± 16 pixels for a target macroblock of 16 pixels vertically and horizontally, the entire search range is 48 pixels vertically and horizontally, and data transfer required to search for a motion vector for one target macroblock The amount is 48 × 48 = 2304 bytes.

  In the tracking method and the gradient method, although the amount of data transferred at one time is small, it is necessary to increase the number of searches in order to increase the search accuracy. The amount of data transfer increases in proportion to the number of searches, and the frame memory 80 transfers to the SRAM 66. The data transfer becomes a bottleneck of the motion detection process and the processing performance decreases. In the full search method, data transfer is performed only once. However, since pixel data in the entire search range is transferred from the frame memory 80 to the SRAM 66, it takes time to transfer the data, which also affects the processing performance.

  In the tracking method and the gradient method, pixel data to be referred to is transferred from the frame memory 80 for each search while changing the position of the macroblock to be referred to in the reference image in units of integer pixels or decimal pixel precision. Most of them overlap with the pixel data referred to in the previous search, and it is wasteful to transfer the pixel data to be referred again every time the search is performed. Further, in the full search method, pixel data in the entire search range is transferred from the frame memory 80 to the SRAM 66. Therefore, pixel data in the search range having an extremely low probability of reference is transferred, which is also wasteful in data transfer. .

  As described above, when detecting a motion vector, generally, data transfer from the frame memory 80 to the SRAM 66 is likely to be wasted, and the bandwidth of the SBUS 82 is compressed, which easily becomes a bottleneck of the encoding process. Based on this awareness of problems, the present applicant has come to recognize that there is room for improvement regarding data transfer from the frame memory 80. Hereinafter, configurations and operations of some motion compensation units 60 for improving data transfer from the frame memory 80 will be described.

  FIG. 3 is a diagram illustrating the configuration of the motion compensation unit 60. Constituent elements common to the general configuration of FIG.

  In order to hold the pixel data transferred from the frame memory 80, the motion compensation unit 60 includes a prefetch SRAM 64 in addition to the normal SRAM 66. The motion vector detection unit 62 transfers data of a pixel region having a high reference frequency from the frame memory 80 to the prefetch SRAM 64 in advance within a predetermined search range of the reference image held in the frame memory 80. On the other hand, the motion vector detection unit 62 transfers pixel data that needs to be referred to when searching for a motion vector from the frame memory 80 to the SRAM 66 each time the search is performed. However, the pixel data already held in the prefetch SRAM 64 is not transferred from the frame memory 80 to the SRAM 66, and the pixel data held in the prefetch SRAM 64 is preferentially used.

  The motion vector detection unit 62 reads pixel data from the prefetch SRAM 64 if the pixel data to be referred to in each search for the target macroblock is in the prefetch SRAM 64, and if not in the prefetch SRAM 64, selects the pixel data to be referred to. It is transferred from the frame memory 80 to the SRAM 66 and used.

  When starting the motion vector search for the next new target macroblock, the motion vector detection unit 62 transfers pixel data having a high reference frequency in the motion search of the new target macroblock from the frame memory 80 to the prefetch SRAM 64 in advance. Then, the data held in the prefetch SRAM 64 is updated.

  The motion vector detection unit 62 can refer to the pre-reading SRAM 64 and the SRAM 66 by switching, thereby preventing repeated transfer of duplicate data from the frame memory 80 and reducing the amount of data transferred from the frame memory 80. For example, suppose that the pre-reading SRAM 64 holds a pixel area of 40 pixels in the vertical and horizontal directions as pre-read data, and the search for macroblocks of 16 pixels in the vertical and horizontal directions is repeated six times in the motion search. If all the pixels to be referred to in the six searches are in the prefetch SRAM 64, the data transfer amount is only one data transfer to the prefetch SRAM 64. The amount of data transfer required to search for a vector is 40 × 40 = 1600 bytes.

  As described above, the motion vector detection unit 62 reads pixel data that is highly likely to be referred to a plurality of times in the search for the motion vector of the target macroblock in advance from the frame memory 80 and transfers the pixel data to the prefetch SRAM 64. The data to be transferred to is determined by the following criteria.

(1) Determination based on motion vectors of surrounding macroblocks When there is a motion vector that has already been detected in the surrounding macroblocks of the target macroblock, the motion vector detecting unit 62 determines the motion vectors of the surrounding macroblocks. Are predicted to be repeatedly referred to in the motion search of the target macroblock, and are transferred to the prefetch SRAM 64 in advance.

(2) Determination based on the motion vector of the entire screen When the entire screen is moving by scrolling the screen or the like, the motion vector detection unit 62 determines that the pixel area of the movement destination expected by the scroll has a high reference frequency. Then, the data is transferred to the prefetch SRAM 64 in advance.

(3) Determination based on the motion vector of the macroblock at the same position in the past or future frame The motion vector detection unit 62 determines the motion vector of the reference macroblock at the same position as the target macroblock in the past or future frame. A pixel region that is frequently referred to in the motion search of the target macroblock is predicted and transferred to the prefetch SRAM 64.

  This is particularly effective when the motion of the image has a correlation in the time axis direction, and the motion of the target macroblock of the encoding target frame can be predicted to some extent from the motion vectors of the frames ahead of time. For example, when following a linear motion model, the search range can be narrowed down by linearly predicting the motion.

  Here, the target macroblock and the reference macroblock do not necessarily have to be at the same position on the image. For example, since the pixel position may change due to screen scrolling or the like, the target macroblock and the reference macroblock may be in a corresponding relationship even if the positions on the image are different. In such a case, a pixel region that is frequently referred to in the motion search of the target macroblock may be predicted from the motion vector of the reference macroblock corresponding to the target macroblock.

(4) Fixed to a peripheral region centered on the target macroblock The motion vector detection unit 62 may repeatedly refer to a range expanded by a predetermined number of pixels in the vertical and horizontal directions around the target macroblock. It is determined that the value is high, and is transferred to the prefetch SRAM 64. This is because when the movement is not so intense, there is a high probability of matching within a narrow range centered on the target macroblock. The vertical and horizontal pixel widths around the target macroblock are adjusted according to the image size, that is, the resolution.

(5) Determination based on search range found in previous pass When encoding is performed over a plurality of passes, the motion vector detection unit 62 repeatedly searches the current pass for the search range found in the previous pass. It is determined that there is a high possibility of being read, and it is transferred to the prefetch SRAM 64.

  The size of the area occupied by the pixel data to be transferred in advance to the pre-reading SRAM 64 may be variable. Depending on factors such as the amount of motion and the accuracy required for motion search, it may be necessary to dynamically change the size of the pixel region to be searched. In addition, because of restrictions on the design of the device on which the encoding device 100 is mounted, the capacity of the prefetch SRAM 64 may be limited, and depending on the device, an SRAM can be added. It is also necessary to configure the pixel data area size so that it can be changed as appropriate according to the capacity limit and usage limit of the prefetch SRAM 64. The size of prefetched data is determined according to the following criteria.

(1) Based on the motion vectors of surrounding macroblocks, the size of the search range and the degree of variation are determined, and the size of prefetched data is determined.
(2) The size of prefetched data is determined based on the motion vector of the entire image such as screen scrolling.
(3) The size of the search range is determined based on the motion vector of the macroblock at the same position in the past or future frame, and the size of the prefetched data is determined.
(4) Using the input information from the outside, for example, the shooting mode, it is determined whether or not the image is a moving image with a large amount of motion, and the size of prefetched data is determined.

  FIG. 4 is a diagram illustrating another configuration of the motion compensation unit 60. In this configuration, the storage area of the prefetch SRAM 65 is divided into a plurality of tiles, and data can be updated in tile units. The motion vector detection unit 62 transfers a pixel area having a high reference frequency among the reference images held in the frame memory 80 to each tile of the prefetch SRAM 64.

  The motion vector detection unit 62 updates the data in the prefetch SRAM 65 when starting a motion vector search for the next new target macroblock. In the configuration of FIG. 4, the data update is performed in units of tiles.

  When there is a relationship between the previous target macroblock and the current target macroblock, the search ranges of both macroblocks often overlap, and pixel regions having a high reference frequency are often common. Therefore, the motion vector detection unit 62 does not discard data frequently referenced in the motion search of the current target macroblock among the pixel data referred to in the previous motion search of the target macroblock, and stores it in the prefetch SRAM 64. Leave it in tile units and reuse it for motion search of the current target macroblock.

  The motion vector detection unit 62 uses pixel data with high reference frequency in the previous motion search of the target macroblock. For pixel data that does not have high reference frequency in the current motion search of the target macroblock, the motion vector detection unit 62 performs prefetching on a tile basis. In the tile discarded from the SRAM 64 and the data discarded, pixel data having a high reference frequency is transferred from the frame memory 80 and held in the current motion search of the target macroblock.

  In the frame memory 80, the pixel position on the image and the position in the storage area correspond to each other. However, since the prefetch SRAM 65 replaces data in units of tiles, the pixel position on the image and the storage area are stored in the prefetch SRAM 65. There is no corresponding relationship as in the frame memory. Therefore, a correspondence table for storing the correspondence between the pixel position on the image and the position in the storage area is provided.

  When the address conversion unit 67 receives the reference address designating the pixel position from the motion vector detection unit 62, the address conversion unit 67 refers to the correspondence table and uses the reference address from the motion vector detection unit 62 as the actual storage destination of the prefetch SRAM 65. The real address is converted to the designated address, and the real address is supplied to the prefetch SRAM 65. The prefetch SRAM 64 outputs the pixel data stored at the real address designated by the address conversion unit 67 to the motion vector detection unit 62.

  By using the prefetch SRAM 65 that can update data in units of tiles, if there is an overlap in the search range between two target macroblocks to be processed in succession, the pixel data in the overlapped range is not discarded and the next The target macroblock can be reused, the amount of data transferred from the frame memory 80 can be further reduced, and the bandwidth consumption of the SBUS 82 can be suppressed.

  FIG. 5 is a diagram illustrating still another configuration of the motion compensation unit 60. In this configuration, a first prefetch SRAM 64a and a second prefetch SRAM 64b are provided. The first prefetch SRAM 64a is used to transfer pixel data having a high reference frequency in the target macroblock currently being processed, and the second prefetch SRAM 64b has a reference frequency in the target macroblock to be processed next. Used to transfer high pixel data in advance.

  Thereby, when the processing of the next target macroblock is started, since the pixel data has already been prefetched in the second prefetch SRAM 64b, the motion vector search can be started immediately. Further, based on the motion search result of the current target macroblock, the pixel data to be prefetched in the next target macroblock is determined and transferred to the second prefetch SRAM 64b. If there is data to be used in the next target macroblock, the data can be copied from the first prefetch SRAM 64a to the second prefetch SRAM 64b.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above description, the prefetch SRAM 64 and the normal SRAM 66 are physically provided separately. However, one SRAM is logically divided into a prefetch memory area and a normal memory area to have respective functions. Also good.

It is a block diagram of the encoding apparatus which concerns on embodiment. It is a figure explaining the general structure of the motion compensation part of FIG. It is a figure explaining the structure of the motion compensation part of FIG. It is a figure explaining another structure of the motion compensation part of FIG. It is a figure explaining another structure of the motion compensation part of FIG.

Explanation of symbols

  20 DCT section, 30 quantization section, 60 motion compensation section, 62 motion vector detection section, 64, 65 prefetch SRAM, 66 SRAM, 67 address conversion section, 68 motion compensation prediction section, 80 frame memory, 82 SBUS, 90 variable A long encoding unit, 100 encoding apparatus.

Claims (9)

An encoding device for encoding a frame of a moving image,
A frame memory that holds a reference frame to be referred to when motion detection of the target block of the encoding target frame is performed;
A motion detector that refers to the reference frame stored in the frame memory and performs motion detection by repeating motion search,
The motion detection unit includes a prefetch memory that transfers pixel data frequently referenced during motion search from the frame memory and holds the pixel data in the reference frame. Device.   The encoding apparatus according to claim 1, wherein the prefetch memory is provided in the same integrated circuit as the motion detection unit.   In addition to the prefetch memory, the motion detection unit has a normal memory that transfers and holds reference pixel data from the frame memory each time a motion search is performed, and the motion detection unit holds the prefetch memory in the prefetch memory The encoding device according to claim 1, wherein the reference pixels that are not used are stored and referred to in the normal memory. The prefetch memory is configured such that the storage area is divided into a plurality of tiles, and data can be updated in units of tiles.
When the motion detection unit starts motion detection of the target block, the motion detection unit does not discard the pixel data referred to in the previous motion search of the target block stored in at least some of the tiles. The encoding apparatus according to claim 1, wherein the encoding apparatus is reused for searching.   5. The pixel data that is frequently referred to and that is transferred from the frame memory to the prefetch memory in advance is pixel data of a peripheral region centered on the position of the target block. The encoding apparatus in any one.   The frequently referred pixel data transferred from the frame memory to the prefetch memory in advance is pixel data of a region corresponding to a prediction block obtained by motion detection for other blocks around the target block. The encoding apparatus according to claim 1, wherein the encoding apparatus is provided.   The frequently-referenced pixel data transferred from the frame memory to the prefetch memory in advance is predicted by motion detection for a reference block at a position corresponding to the target block in a past or future frame of the moving image. 5. The encoding apparatus according to claim 1, wherein the encoding data is pixel data of a region to be processed.   8. The size of an area occupied by the frequently referenced pixel data transferred from the frame memory to the prefetch memory in advance is set to a different value depending on the target block. An encoding device according to claim 1.   From a frame memory that holds a reference frame to be referred to when motion detection is performed on a target block of an encoding target frame of a moving image, pixel data having a high reference frequency is previously stored in the pixel data of the motion search range in the reference frame. If the pixel data to be transferred and stored in the prefetch memory is in the prefetch memory, the pixel data held in the prefetch memory is referred to. If not, the frame data is stored in the prefetch memory. An encoding method, wherein motion search is performed by referring to pixel data held in a memory.

Images