JP2015005903A - Compressor, decompressor and image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a compressor which is suitable for multimode in an integrated compression system and capable of achieving high compressibility, a decompressor and an image processing apparatus.SOLUTION: The compressor comprises: pixel value prediction means for calculating a prediction value of each pixel in accordance with each prediction mode while preparing a plurality of prediction modes formed by combining copy prediction of neighboring pixels and average prediction of neighboring pixels with respect to the pixels within a pixel block excluding a corner origin pixel; residual calculation means for calculating a residual with respect to the pixels within the pixel block; residual encoding means for encoding the residual of the pixels; prediction mode selection means for calculating a residual bit size of the pixel block encoded by the residual encoding means and selecting a selection prediction mode minimizing the residual bit size of the encoded pixel block with respect to the prediction modes; and symbol code generation means for generating encoded frame data by formatting information of the pixel block including a pixel value of the origin pixel, the selection prediction mode and residual information.

Description

  The present invention relates to an image signal compressor, decompressor, and image processing apparatus used when buffering image information in a video codec system or the like.

  HEVC, H.M. In a video codec system including encoders and decoders such as H.264 / AVC and MPEG-2, a large-capacity external DRAM is usually required to buffer a large amount of data such as a reference frame. As a result, a wide bandwidth is required for data transfer to and from the memory, and power consumption for memory access accounts for a large part of the power consumption of the entire system. Thus, solving this memory bandwidth problem is an important technical challenge.

  Various viewpoints such as technology for reusing reference frame data that has been over-called at various levels, and technology for optimizing DRAM access efficiency by optimizing the memory controller architecture Research has been done. Among them, embedded compression (EC) known as reference frame recompression is a means for solving the above-mentioned DRAM bandwidth problem. In EC, to reduce memory data traffic, it is inserted as an additional layer between the codec core and the DRAM controller, compressing the reference frame before storing it in the DRAM and fetching the data back from the DRAM. Extract (decompress) before starting.

  Conventional EC schemes can be broadly classified into two categories. One is irreversible EC with a fixed compression ratio (CR). If the fixed CR is used, the image quality is inevitably lowered, and error propagation caused by the quality degradation of the reference frame becomes a serious problem. The other is lossless EC. The lossless EC scheme needs to have a variable compression rate. Therefore, a store and fetch mechanism for the partition size of compressed data is required as a dedicated memory mechanism.

  Usually, data compression in lossless EC consists of a prediction stage and an entropy coding stage. In the prediction stage, spatial domain prediction and frequency domain prediction are widely used. In the entropy coding stage, variable length coding (VLC) is widely used, but hardware implementation is very difficult to achieve high throughput VLC. Therefore, two-step entropy coding is adopted in recent EC schemes. In this method, residuals are first grouped into small groups, and then fixed-length coding is performed based on the characteristics of local residuals in each group.

  In the conventional spatial domain prediction algorithm in the embedded compression, the hierarchical average and copy prediction (HACP) described in Non-Patent Document 1 shows excellent performance. FIG. 18 is a diagram illustrating the principle of HACP for an 8 × 8 pixel block (cited from Non-Patent Document 1). In FIG. 18, each level 1 cell represents an original pixel. Each square of level 2 represents an average value of each block obtained by dividing each pixel of level 1 into 16 blocks of 2 × 2 pixels. Each square of level 3 represents an average value of each block obtained by dividing each pixel of level 2 into 4 blocks of 2 × 2 pixels. The base pixel is an average value of four pixels at level 2.

At level 3, the base pixel is used as the predicted value. At level 2, for the pixels 101 to 104 in FIG. 18, the pixel value of each corresponding pixel at level 3 is used as the predicted value. For the other pixels, the pixel at the base point of the arrow shown in FIG. 18 is used as the prediction source pixel. For a pixel to which two arrows are directed, the average value of the pixels at the base point of each arrow is used as the predicted value (this is referred to as “average prediction”). For the pixel to which one arrow is directed, the pixel value of the pixel at the base point of each arrow is used as the predicted value (this is called “copy prediction”). At level 1, the pixel values of the corresponding pixels at level 2 are used as predicted values for the pixels 105 to 108 in FIG. For the other pixels, the pixel at the base point of the arrow shown in FIG. 18 is used as the prediction source pixel. In level 1, the same average prediction and copy prediction as in level 2 are performed. When the level of each layer is L (= 1, 2, 3), the distance between the prediction source pixel at level L and the original pixel that is the prediction target is 2 ^L−1 . In each of the levels 1, 2, and 3, the pixel data is divided into the pixel value of the base pixel and the residual of each pixel in each layer by obtaining a residual obtained by subtracting the predicted value from the pixel value of each square. Data is compressed after conversion.

  In HACP, (1) horizontal average prediction (HAP) using an average of left and right pixels as a predicted value, and (2) vertical average prediction (VAP) using an average of upper and lower pixels as a predicted value. (3) horizontal copy prediction (HCP) using the pixel value of the right or left adjacent pixel as a predicted value, and (4) vertical copy prediction using the pixel value of the upper or lower adjacent pixel as a predicted value ( vertical copy prediction (VCP) is used. The residual calculation by each average prediction and each copy prediction is calculated by the following equation.

Here, p _{i, j} is a pixel value, x _{i, j} is a prediction residual of the pixel p _{i, j} , and d is a distance between the prediction pixel and the reference pixel.

J. Kim and C. M. Kyung, "A lossless embedded compression using significant bit truncation for HD video coding", IEEE Trans. Circuits Syst. Video Techn., Vol.20, no.6, pp. 848-860, 2010. D.Zhou, J. Zhou, X. He, J. Zhu, J. Kong, P. Liu, and S. Goto, "A 530 Mpixels / s 4096x2160 @ 60fps H.264 / AVC High Profile Video Decoder Chip," IEEE J. Solid-State Circuit, vol.6, no.4, pp.777-788, 2011.

  The inventor first examined the spatial correlation between adjacent pixels in a natural sequence (natural image). FIG. 19 is a diagram illustrating a prediction method (a) and its transposition (b) in level 1 of the HACP scheme. In the HACP method, an average value of two adjacent pixels is mainly used as a predicted value. However, for some pixels that cannot be predicted from the pixel values of adjacent pixels, prediction using a farther pixel is performed. For example, a pixel predicted at level 2 (“Predicted from level-2” in FIG. 19) is predicted using pixels that are two pixels apart, and a pixel predicted at level 3 (“Predicted from level in FIG. 19). -3 ") is predicted using pixels that are four pixels apart.

  However, in general, the spatial correlation between the pixel values of pixels at a distant distance is much smaller than the spatial correlation between the pixel values of adjacent pixels. FIG. 20 is a diagram illustrating a probability distribution of residuals in the HACP scheme for different prediction distances. In FIG. 20, the horizontal axis represents the residual value (HAC residual), and the vertical axis represents the distribution probability. HAC_dist.1 represents prediction based on neighboring pixels, HAC_dist.2 represents prediction based on pixels that are two pixels apart, and HAC_dist.4 represents prediction based on pixels that are four pixels apart. The entropy of the HAC residual calculated by the prediction by the adjacent pixels is 3.7677, but increases to 4.7594 in the prediction by the two-pixel distance, and increases to 5.23 in the prediction by the four-pixel distance. Therefore, it can be seen that HACP level 2 and 3 predictions have a significant adverse effect on compression performance.

  In addition to the problem of the predicted distance, there is a problem that the HACP method is not suitable for multimode expansion. FIG. 19B shows a HAC transpose matrix (transposition of the rows and columns in FIG. 19A). FIG. 19B also shows a pixel (“Different prediction”) that is predicted differently from FIG. There are only 20 pixels of different prediction methods between the HACP 8 × 8 prediction block (FIG. 19A) and its transposed block (FIG. 19B). Furthermore, the predicted values for most pixels at level 2 are the same. Therefore, even if the HAC transposition mode (FIG. 19B) is added as the multimode, an improvement in the compression rate cannot be expected.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a compressor and a decompressor that are suitable for multimode and can achieve a higher compression ratio than those using the conventional HACP method. And providing an image processing apparatus using the same.

A first configuration of a compressor according to the present invention is a compressor for compressing frame data of an image frame,
Block dividing means for dividing the input frame data into pixel blocks of a predetermined size;
Consists of a combination of copy prediction of any one adjacent pixel and average prediction of any two adjacent pixels for each pixel in the pixel block excluding the starting pixel at any one corner of the pixel block A plurality of prediction modes, and a pixel value prediction means for calculating a prediction value of each pixel in the pixel block according to each prediction mode;
For each pixel in the pixel block, residual calculation means for calculating a residual obtained by subtracting the predicted value of the pixel from the pixel value of the pixel;
Residual encoding means for encoding the residual of each pixel in the pixel block;
For each prediction mode, the residual bit size of the pixel block encoded by the residual encoding means is calculated, and the encoded residual bit block size of the pixel block is minimized. A prediction mode selection means for selecting the prediction mode as the selected prediction mode;
The information of the pixel block including the pixel value of the starting pixel, the information of the selected prediction mode, and the residual information of the pixel block calculated and encoded in the selected prediction mode is formalized in a predetermined format. Code code generating means for generating encoded frame data;
It is provided with.

  According to this configuration, the pixel value predicting unit predicts the pixel value in the pixel block by copy prediction of adjacent pixels or average prediction of any two adjacent pixels. It is possible to increase the data compression rate without including prediction based on distance. In addition, since it is not a hierarchical encoding method, by preparing a plurality of prediction patterns (prediction modes) consisting of various combinations of copy prediction of any one adjacent pixel and average prediction of any two adjacent pixels. Multimode prediction is possible, and the compression rate can be further improved.

  Here, “copy prediction” refers to a prediction method that uses the pixel value of an adjacent pixel as it is as the prediction value of a prediction target pixel. For copy prediction, “horizontal copy prediction” using the pixel values of pixels adjacent in the horizontal direction (row direction) as prediction values and the pixel values of pixels adjacent in the vertical direction (column direction) are used as prediction values. There is "vertical copy prediction". “Average prediction” refers to a prediction method that uses an average value of pixel values of pixels adjacent to the prediction target pixel in the left-right or vertical direction as a prediction value of the prediction target pixel. For the average prediction, “horizontal average prediction” (equation (2a) to be described later) using the pixel value average of two pixels adjacent in the horizontal direction (row direction) as the predicted value and adjacent in the vertical direction (column direction). There is “vertical average prediction” (formula (2b) described later) using a pixel value average of two pixels as a prediction value.

According to a second configuration of the compressor of the present invention, in the first configuration, the block dividing unit divides the frame data into pixel blocks of n × n size (n is an even number of 8 or more). Yes,
The pixel value predicting means uses the corner pixel in the 0th row and the 0th column as the starting pixel, and the pixels in the pixel block excluding the starting pixel are grouped into an M group that is a group of (n−1) vertical pixels. And the prediction mode when the pixel value prediction means calculates the prediction value is at least:
(A) M group consisting of column pixels in the first to (n-1) th columns in the 0th row of the pixel block, and columns in the first to (n-1) th rows in the (n-1) th column. Horizontal copy prediction for the M group of pixels, vertical copy prediction for each M group of even-numbered columns of pixels in the first to (n-1) th rows, and the first to (n-) except for the (n-1) th column. 1) Prediction mode 0 in which a prediction method of horizontal average prediction is assigned to each M group of odd-numbered column pixels in a row.
(B) Prediction mode 1, which is a transpose of the prediction mode 0,
(C) Vertical copy prediction for the M group consisting of column pixels in the 1st to (n-1) th rows of the 0th column, and horizontal to each M group consisting of column pixels in each row of the 1st to (n-1) th columns. Prediction mode 2 for assigning a prediction method for copy prediction,
(D) Prediction mode 3, which is a transpose of the prediction mode 2,
These four prediction modes are included.

  According to this configuration, if multimode prediction is performed using at least the prediction modes 0 to 3, prediction of any one of the prediction modes 0 to 3 is performed for blocks with various characteristics such as slowly changing blocks and flat blocks. It has been experimentally confirmed that the mode has a high compression efficiency and can obtain a high compression ratio as a whole (see Table 1 described later). In the present invention, other prediction modes can be added in addition to the four prediction modes.

According to a third configuration of the compressor of the present invention, in the second configuration, the residual encoding unit includes:
S group dividing means for dividing the residual of each pixel in the pixel block into S groups of a predetermined size that have the same pixel value prediction method by the pixel value prediction means and in which pixels are adjacent to each other;
For each S group, the absolute value of the maximum value and the minimum value of each residual belonging to the S group are calculated, and the encoding mode of the S group is calculated based on the absolute value of the maximum value and the absolute value of the minimum value Encoding mode determining means for determining
For each of the prediction modes, the pixel block is partitioned into a plurality of L groups having a predetermined size and one M group having a half size, and each L group or M group is divided into the L group or M group. Regrouping is performed by combining the S groups belonging to the same encoding mode, and a regrouping mode is determined for each L group or M group according to the group pattern after regrouping Regrouping means to
SFL encoding means for encoding the residual of each group reorganized by the regrouping means based on a predetermined encoding table according to the encoding mode,
The code code generation means includes the pixel value of the origin pixel, the information of the selected prediction mode, the regrouping mode of each L group or M group, and the pixel block calculated and encoded in the selected prediction mode The pixel block information including the residual information is formalized in a predetermined format.

  According to this configuration, by dividing a pixel block into S groups of a predetermined size that have the same prediction method and pixels adjacent to each other, there is a high probability that pixels that are statistically similar in residual are included in the same group. . Therefore, the compression efficiency of the residual data can be improved by encoding (entropy encoding) based on the encoding table for each group. Further, the coding table applied to each S group is classified according to the coding mode by the absolute value of the maximum value and the absolute value of the minimum value in the S group, and optimal code allocation is performed. Are combined and regrouped into a larger group, further improving the compression efficiency of the residual data.

According to a fourth configuration of the compressor of the present invention, in the third configuration, the S group dividing unit divides the M group into (n / 2) pixels and (n / 2-1) for each prediction mode. ) It is divided into S groups which are divided into two groups of pixels,
The regrouping means includes
For the prediction mode 0, the odd-numbered M group is divided into M group pairs composed of two M groups, and each M group pair is defined as one L group, and the even-numbered M group is defined as two M groups. Each L group is formed by dividing each M group pair into one L group by dividing into M group pairs consisting of groups,
For prediction mode 2, the M groups in each row of the first to (n-1) th columns are divided into M group pairs composed of two adjacent M groups, and each M group pair is divided into one L group. Thus, each L group is formed.

A decompressor according to the present invention is a decompressor that restores the frame data from the encoded frame data encoded by the compressor having any one of the first to fourth configurations,
Scanning means for sequentially scanning from the base pixel;
For the pixels sequentially designated by the scanning of the scanning means, based on the pixel value of the starting pixel included in the encoded frame data or the pixel value of the selected prediction mode or the already calculated adjacent pixel, Predicted value decoding means for calculating a predicted value of the value;
Residual decoding means for calculating a pixel value of the pixel based on the predicted value of the pixel calculated by the predicted value decoding means and residual information of the pixel included in the encoded frame data;
It is provided with.

An image processing apparatus according to the present invention includes a codec core that encodes or decodes a moving image by a predetermined codec method, and
A frame memory that temporarily stores frame data used by the codec core for encoding or decoding operations, and an image processing apparatus comprising:
Between the codec core and the frame memory, the frame data output from the codec core is compressed and the encoded frame data is stored in the frame memory. A memory interface that reads out the encoded frame data from the frame memory when there is a request to read out the frame data, decodes it into the frame data, and outputs the frame data to the codec core;
The memory interface is
The compressor of any one of the first to fourth configurations that compresses the frame data into the encoded frame data;
And a decompressor according to the present invention for decoding the encoded frame data into the frame data.

  As described above, according to the present invention, the compressor and the decompressor used in the built-in compression method are suitable for the multimode and can achieve a higher compression ratio than that using the conventional HACP method. A compressor and a decompressor, and an image processing apparatus using the compressor and decompressor can be provided.

It is a figure showing the prediction mode and group division used in a MDA scheme. It is a figure which shows the example which groups the residual of one prediction block of 8x8 pixels into M group and S group in PMode0. It is a figure showing regroup mode. (A) Regroup mode in M group, (b) Regroup mode in L group. 1 is a diagram illustrating an overall configuration of an image processing apparatus according to a first embodiment. It is a figure showing the internal structure of the compressor 4 of FIG. It is a figure showing the internal structure of the decompressor 5 of FIG. 6 is a flowchart illustrating a frame compression operation by the compressor 4 of the image processing apparatus according to the first exemplary embodiment. 3 is a diagram illustrating a data format of pixel block data compressed by a compressor 4 in the image processing apparatus of Embodiment 1. FIG. It is a flowchart which shows the other example of block data compression. 6 is a flowchart illustrating a frame expansion operation by the decompressor 5 of the image processing apparatus according to the first exemplary embodiment. It is a figure which shows the hardware mounting example of the compressor 4. FIG. It is a figure showing the timing of the pipeline processing cycle of the compressor 4 of FIG. It is a figure which shows the hardware mounting example of the decompressor 5. FIG. It is a figure showing the timing of the pipeline processing cycle of the decompressor 5 of FIG. It is a figure showing the prediction mode and group division | segmentation used in the MDA scheme with respect to a 16x16 pixel block. It is a figure which shows the example which groups the residual of one prediction block of 16x16 pixels into S group in PMode0. In PMode0, it is a figure which shows the other example in the case of grouping the residual of one prediction block of 8x8 pixel into S group. It is the figure which showed the principle of HACP with respect to an 8x8 pixel block. It is a figure showing the prediction method (a) in the level 1 of a HACP system, and its transposition (b). It is a figure showing the probability distribution of the residual in a HACP system with respect to different prediction distance.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings.

[1] Basic prediction / coding scheme (1) Multi-mode DPCM (differential pulse modulation) and averaged prediction Average prediction using an average value of two adjacent pixels as a predicted value is an average of one adjacent pixel. The prediction accuracy is superior to copy prediction (directly replicate prediction) in which the value is a predicted value. However, as described above, in the HACP method, for some pixels, prediction is necessarily performed using pixel values of pixels that are two or four pixel distances apart, and these pixel values may cause a decrease in compression performance. It is the cause. In other words, there is a conflict between taking a better prediction mode or taking a shorter prediction distance. Therefore, in the present invention, an MDA (multi-mode DPCM scanning and averaging) scheme newly devised this time is used. Hereinafter, the MDA scheme will be described.

  FIG. 1 is a diagram illustrating a prediction mode and group division used in an MDA scheme. FIGS. 1A, 1B, 1C, and 1D represent prediction modes PMode0, PMode1, PMode2, and PMode3, respectively.

  PMode0 is a basic form of the prediction method in the MDA scheme. The original pixel value is given to the upper left pixel. The prediction of the pixels in the 0th row excluding the upper left one pixel is performed by horizontal DPCM (horizontal copy prediction). Prediction of 28 pixels in even columns in the first to seventh rows is performed by vertical DPCM (vertical copy prediction). Prediction of pixels in columns 1, 3, and 5 in the first to seventh rows is performed by horizontal average prediction of left and right adjacent pixels. The prediction of the pixel in the last column 7 in the first to seventh rows is performed by duplicating the value of the left adjacent pixel (that is, horizontal copy prediction). Thus, since PMode 0 uses only the pixel values of adjacent pixels when performing prediction, the problem of deterioration in prediction performance due to pixels of 2 pixel distance or 4 pixel distance in the HACP method is avoided.

Here, the horizontal average prediction and the vertical average prediction are defined by the following expressions (2a) and (2b), respectively. p _{i, j} is the pixel value of the pixel (i, j), and Pr _{i, j} is the predicted value of the pixel (i, j).

  Natural sequences include blocks with various features, such as slowly changing blocks and flat blocks, so generally one prediction mode cannot be suitable for these various feature blocks. . Therefore, in order to deal with various image features, multi-mode prediction (PMode0 to PMode3) is applied in the MDA scheme.

  PMode1 shown in FIG. 1B is a transposition of PMode0, and has 43 pixels with different prediction schemes with respect to PMode0. In PMode1, vertical average prediction and horizontal DPCM are used.

  However, since the average prediction in PMode0 and PMode1 and the direction of DPCM are orthogonal, these two modes are not suitable for slowly changing blocks. Therefore, in the MDA scheme, simple horizontal DPCM and vertical DPCM scanning modes such as PMode2 and PMode3 are added (FIGS. 1C and 1D). In the multi-mode, a prediction mode with the minimum compression rate is selected from the four prediction modes for each block.

  In order to verify the validity of the multimode shown in FIG. 1, the probability of the above four prediction modes being selected using 18 test sequences was investigated. Table 1 shows the selected probabilities for the four MDA prediction modes. In Table 1, Qp represents a quantization parameter. The probability value in the table is an average value of all frames of the 18 test sequences.

  From the above experimental results, it is understood that each prediction mode of PMode0 to PMode3 is selected with a probability of approximately 20% to 30%, and it is reasonable to adopt four prediction modes of PMode0 to PMode3. Therefore, in the MDA scheme, a combination of these four prediction modes is adopted.

(2) Grouping of residuals in MDA In order to improve the compression ratio of the residual of each pixel obtained by prediction, truncation (SBT: significant bits truncation) and semi-fixed length (SFL) It is necessary to classify the residuals before encoding. By adopting an efficient residual grouping scheme, higher compression performance can be obtained. Therefore, in order to adapt to the characteristics of SFL coding and SBT coding, those showing similar distributions are grouped as one group. For this purpose, it is generally considered that the residuals predicted by the same prediction method should be classified into the same group.

  For example, according to the averaged prediction and DPCM assignment in PMode 0 in FIG. 1A, one prediction block of 8 × 8 pixels can be divided into nine groups shown in FIG. These nine groups are referred to as “middle size groups” (M groups). To make the group size the same, all M groups are composed of 7 pixels.

  However, in an actual sequence, there are cases where only the residual of one or two pixels in one M group is not larger than (or very small). In such a case, it is more efficient to group into two small size groups (S group) consisting of 3 pixels and 4 pixels than to group the residuals into one M group. FIG. 2B shows an example of grouping into S groups in PMode0.

  On the contrary, in a block in which the pixel value changes smoothly, the encoding mode CM in SFL encoding is similar in the block. In such a case, it is more efficient to combine two M groups into one large size group (L group). 1A to 1D show a method for grouping L groups in each prediction mode. 1A to 1D, pixels with the same color belong to the same L group. In each mode of FIG. 1, the blocks are grouped into one M group (group 0) and four L groups (groups 1 to 4).

  Residual compression efficiency can be improved by variously combining the three basic residual groups (S group, M group, and L group) as described above.

(2) Regrouping of residuals If the S group is used, the number of bits for representing the residual can be reduced as compared with the case where the M group or the L group is used. However, when the S group is used, the overhead of the encoding mode CM is twice or more that of the M group and 3.6 times that of the L group. Therefore, in order to reduce the additional bits of the S group, the present invention employs a scheme for regrouping four S groups in one L group.

  In most cases, the coding modes of four S groups (or two S groups) within one L group (or M group) are similar or identical. Therefore, regrouping is performed by combining S groups of the same coding mode.

  As shown in FIGS. 1A to 1D, one 8 × 8 block is divided into four L groups (large groups 1 to 4) and one M group (group 0). Among these, the M group (group 0) is divided into two S groups of a 3-pixel S group and a 4-pixel S group as shown in FIG. 2B. In regrouping, if the coding modes of the two S groups are the same, they are combined into one group. Therefore, for the M group, the two modes (M, 2S) of FIG. Therefore, for the M group, a 1-bit additional flag is sufficient to represent information on the regrouping mode (RGM). Similarly, in the L group (large group 1 to 4), 15 modes (RG0 to RG14) in FIG. 3B are generated by regrouping. In FIG. 3B, pixels of the same color represent the same encoding mode. Therefore, for the L group, an additional flag of 3 bits is required for RG0 and 4 bits otherwise for representing the regrouping mode (RGM).

(3) Semi-fixed length coding In the residual entropy coding, the semi-fixed length (SFL) coding method described in Non-Patent Document 2 is used. In each group, a coding mode (CM) is determined based on the minimum residual value and the maximum residual value. Then, the residual is encoded according to the semi-fixed length encoding table shown in Table 2.

  In Table 2, mm is obtained from the absolute value of the minimum value p and the maximum value q of the residual values in the group by the following equation.

M is a coding mode (CM) number. M is uniquely determined by the value of mm. D is an encoded sequence. In the column D of Table 2, S is a sign bit of the residual (1 if the residual is negative, 0 if positive). S with an overline is a logical inversion value of S. The underlined value indicates that an additional trailing bit T is required, and 2 ^M−1 or −2 ^M−1 (M is an encoding mode number) is added to the residual value in the group. Used to represent its sign when present. If 2 ^M−1 or −2 ^M−1 does not exist in the residual value in the group, the trailing bit T is not added. Since the trailing bit T is added, when the encoding mode M is 1 to 6, the residual range given in the encoding mode is [−2 ^M−1 +1, 2 ^M−1 ] or [2 ^{M −1} , 2 ^M−1 −1].

(Example 1) When the residual in the group is r = {5, 0, −4}, the maximum value of the residual is p = 5 and the minimum value is q = −4. , Mm = 5, M = 4. Therefore, the encoding sequence is
D = {0101,0000,1100}
Since no residual is 8 (= 2 ^M-1 ) or -8 (= -2 ^M-1 ), no trailing bit T is added in this case.

(Example 2) When the residual in the group is r = {8, 0, −4}, the maximum value of the residual is p = 8 and the minimum value is q = −4. Mm = 8 and M = 4. Therefore, the encoding sequence is
D = {1000, 0000, 1100}
It becomes. In this case, since r (1) = 8, the trailing bit T = 0 is added.

(Example 3) When the residual in the group is r = {− 8, 0, −4}, the maximum value of the residual is p = 0 and the minimum value is q = −8. Therefore, mm = 8 and M = 4. Therefore, the encoding sequence is
D = {1000, 0000, 1100}
It becomes. In this case, since r (1) = − 8, the trailing bit T = 1 is added.

(Example 4) When the residual in the group is r = {8, 0, −8}, the maximum value of the residual is p = 8 and the minimum value is q = −8. Mm = 9 and M = 5. Therefore, the encoding sequence is
D = {01,000,00000,11000}
It becomes. In this case, since no residual is 16 (= 2 ^M-1 ) or -16 (= -2 ^M-1 ), the trailing bit T is not added.

[2] Configuration of Image Processing Device (1) Overall Configuration FIG. 4 is a diagram illustrating the overall configuration of the image processing device according to the present embodiment. In FIG. 4, the image processing apparatus is composed of three parts: a codec core 1, a DRAM 2, and a DRAM interface unit 3. Cordic Core 1 is HEVC, H. This is a part that performs video codec including encoders and decoders such as H.264 / AVC and MPEG-2. In the present invention, the codec core 1 is not particularly limited, and various types (for example, codec cores such as HEVC, H.264 / AVC, and MPEG-2) can be used. Is omitted. The DRAM 2 is a memory for buffering image data such as a reference frame used in the video codec in the codec core 1. The DRAM interface unit 3 is provided as an additional layer between the codec core 1 and the DRAM 2. The DRAM interface unit 3 compresses image data such as a reference frame output from the codec core 1 before storing it in the DRAM 2. When the image data is fetched back to the codec core 1, it is expanded (decompressed). The DRAM interface unit 3 includes a compressor 4, a decompressor 5, a cache memory 6, and a DRAM controller 7. The compressor 4 performs lossless compression of the image data written to the DRAM 2. The decompressor 5 develops the image data read from the DRAM 2. The cache memory 6 is a memory for temporarily storing compressed image data that is written to or read from the DRAM 2. The DRAM controller 7 is a controller that controls data writing to the DRAM 2 and data reading from the DRAM 2.

  The DRAM interface unit 3 may be configured in software using a microcomputer or a reconfigurable logic circuit (FPGA or the like), or a dedicated circuit for speeding up is configured and implemented in hardware. Also good.

(2) Compressor FIG. 5 is a diagram illustrating the internal configuration of the compressor 4 of FIG. Note that FIG. 5 is a block diagram for each functional element for the sake of convenience in order to explain the functions of the compressor 4 in an easy-to-understand manner. In actual hardware implementation, it is not necessarily configured as shown in FIG. Is not limited (see, for example, FIG. 11). The compressor 4 includes a prediction mode table 11, a residual memory 12, a residual calculation unit 13, a frame input unit 14, a frame buffer 14a, a block division unit 15, a block buffer 15a, an MDA prediction unit 16, a prediction mode selection unit 17, a code A generation unit 18, a residual encoding unit 19, and an output unit 20 are provided.

  The prediction mode table 11 stores patterns of the four prediction modes PMode0 to PMode3 shown in FIG. The residual memory 12 is a part that temporarily stores a residual calculated by subtracting the predicted value of the pixel from each pixel of each input image block (8 × 8 pixel image block). The frame input unit 14 is an interface part to which frame data (image data such as a reference frame) output from the codec core 1 is input. The frame buffer 14 a is a memory that temporarily stores frame data input from the frame input unit 14. The block dividing unit 15 divides the frame data input from the frame input unit 14 into pixel blocks of a predetermined size (8 × 8 pixels). The block buffer 15a is a memory that temporarily stores the pixel block cut out by the block dividing unit 15. The MDA prediction unit 16 calculates a prediction value of each pixel of the pixel block in each prediction mode of the prediction modes PMode0 to PMode3 by the above-described MDA (multimode DPCM and averaged prediction) method. The prediction mode selection unit 17 selects the prediction mode with the highest compression efficiency when the pixel block is compressed using the prediction modes in the prediction modes PMode0 to PMode3. The residual calculation unit 13 calculates the residual of the pixel block using the prediction value predicted by the MDA prediction unit 16 and stores it in the residual memory 12. The code generation unit 18 performs format shaping of the encoded pixel block according to the prediction mode selected by the prediction mode selection unit 17. The residual encoding unit 19 reads out the residual value of the pixel block from the residual memory 12, and performs the above-described residual grouping process, regrouping process, and semi-fixed length encoding process. The output unit 20 outputs the encoded pixel block data output from the code generation unit 18 to the cache memory 6.

  5 includes a grouping table 21, a small group dividing unit 22, a CM & T determining unit 23, a regrouping unit 24, and an SFL encoding unit 25. The grouping table 21 stores a coupling pattern (for example, see FIGS. 1 and 2) when grouping into an S group, an M group, and an L group. The small group dividing unit 22 divides the residual according to each prediction mode into S groups. The CM & T determination unit 23 determines the encoding mode (CM) and trailing bit (T) of each S group. The regrouping unit 24 regroups the S groups by the above-described residual regrouping process for each S group divided by the small group dividing unit 22. The SFL encoding unit 25 performs entropy encoding on the residual of each group regrouped by the regrouping unit 24 by the above-described SFL encoding method.

(3) Decompressor FIG. 6 is a diagram showing the internal configuration of the decompressor 5 of FIG. Note that FIG. 6 is a block diagram for each functional element for the sake of convenience in order to explain the functions of the decompressor 5 in an easy-to-understand manner. In actual hardware implementation, it is not necessarily configured as shown in FIG. (For example, refer to FIG. 13). The decompressor 5 includes a prediction mode table 30, a grouping table 31, an input unit 32, a predicted value decoding unit 33, a residual decoding unit 34, a frame restoration unit 35, and an output unit 36. Note that the prediction mode table 30 and the grouping table 31 are the same as the prediction mode table 11 and the grouping table 21 in FIG. 5, and can be the same as those in FIG. The input unit 32 receives data of encoded pixel blocks read from the DRAM 2. The predicted value decoding unit 33 decodes encoded predicted value data among the data input from the input unit 32. The residual decoding unit 34 decodes encoded residual data among the data input from the input unit 32. The frame restoration unit 35 restores each pixel block from the prediction value and the residual decoded by the prediction value decoding unit 33 and the residual decoding unit 34, and synthesizes frame data. The output unit 36 outputs the frame data generated by the frame restoration unit 35 to the codec core 1.

(4) Data compression operation The data compression operation of the image processing apparatus of the present embodiment configured as described above will be described below. FIG. 7 is a flowchart showing the frame compression operation by the compressor 4 of the image processing apparatus of this embodiment. FIG. 8 is a diagram illustrating the data format of the pixel block data compressed by the compressor 4 in the image processing apparatus of the present embodiment.

(4.1) Overall Operation First, frame data FL is input from the codec core 1 to the compressor 4 (S100).

Compressor block dividing unit 15 of the 4 divides the frame data FL entered into pixel blocks _{{B k}} of _{N B} number of 8 × 8 pixels (S101).

Next, the compressor 4, for each pixel block _{B k,} performs block data compression to produce a compressed pixel block data _{B 'k (S102, S103)} .

Finally, the compressor 4 shapes the compressed pixel block data B ′ _k into the format of FIG. 8 and outputs it to the cache memory 6.

(4.2) Block Data Compression Block data compression in step S103 is performed as follows according to the flow of FIG.

First, MDA prediction calculation, residual calculation, residual regrouping, and block cost calculation are executed for each prediction mode of PMode0 to PMode3 in FIG. 1 (S200 to S206). That is, for the prediction mode PMode i (i = 0, 1, 2, 3), the MDA prediction unit 16 first removes the upper left pixel according to the prediction pattern shown in FIG. 1 stored in the prediction mode table 11. A predicted pixel value is calculated for each pixel (S201). The residual calculation unit 13 calculates the residual of each pixel using the calculated predicted value (S202). Here, the residuals are grouped into S-groups of 3 pixels or 4 pixels by the small group dividing unit 22 as illustrated in FIG. Next, the CM & T determining unit 23 calculates mm for each S group by the equation (3), and determines the encoding mode M of each S group according to Table 2 (S203). Then, the regrouping unit 24 performs regrouping by combining the S groups belonging to the same L group or M group and having the same encoding mode M, and FIG. 3B illustrates the L group. The regrouping mode (RGM) of any of RG0 to RG14 is determined, and for the M group, the regrouping mode (RGM) of either M or 2S of FIG. 3A is determined (S204). Then, the SFL encoding unit 25 performs SFL encoding on the regrouped residuals, and generates residual data (encoded residual data) D (PMode i) of the encoded pixel block ( S205). Here, the prediction mode selection unit 17 calculates the block cost BC _i of the encoded pixel block (S206). Here, the block cost BC _i is specifically calculated according to the following equation.

Here, F, PM, RGM, CM, D, and T represent the pixel value of the upper left corner pixel of the block, the prediction mode, the regrouping mode, the encoding mode, the encoding residual data, and the trailing bit, respectively. . ‖X‖ represents the bit cost (number of bits) of X. Here, since the pixel value of each pixel is 8 bits, ‖F‖ = 8. Since the prediction mode ranges from 0 to 3 as shown in FIG. 1, ‖PM‖ = 2. The regrouping mode RGM has a range of 0 to 14 for L group and 0 to 1 for M group as shown in FIG. 3, and there are 4 L groups and 1 M group per block. Is required. However, since RG0 in FIG. 3 is assigned a 3-bit flag (000) and RG1-RG14 is assigned a 4-bit flag (0010-1111), ‖RGM‖ is between 13-17 bits. Value. ‖CM‖ depends on the total number of block groups _{N g.} The number of groups N _g will vary with the M group and regrouping mode of each L group. According to Table 2, the CM per group is represented by 3 bits, so ‖CM‖ = 3N _g and takes a value in the range of 15 to 54. ‖D‖ is the sum of the bit costs ‖D (S-group) ‖ of the S group in the block. From FIG. 2, the number of pixels in one S group is 3 or 4, and from Table 2, the encoding residual data per pixel is M bits when the encoding mode M is less than 7, M = 7 In this case, since it is 8 bits, ‖D (S-group) ‖ can be calculated as shown in Equation (4). Since trailing bits are 0 or 1 bit per group, ‖T‖ takes a value between 0 and 18.

When the arithmetic processing of S200 to S206 is completed for all prediction modes, the prediction mode selection unit 17 selects the prediction mode PMode0 to PMode3 having the smallest block cost BC _i and selects the selected prediction mode. The mode PM is determined (S207).

Next, the code generation unit 18, the pixel values of the upper left pixel of the pixel block B _k, the prediction mode PM, regrouping mode RGM, for each pixel group encoding mode CM, in the prediction mode PM coded pixel blocks Residual data D and trailing bit T are shaped into the data format shown in FIG. 8 to generate a coded block cB _k (S208). In FIG. 8, F is the pixel value of the upper left pixel of the pixel block _Bk (8 bits), PM is the prediction mode (2 bits), and RGM is a regroup of each L block (4) or M block (1). modes (17-bit), CM coding mode (15-54 bits) of each pixel group (5 to 18 pieces), D coding sequence of the residual of each pixel except the upper left pixel of the pixel block _{B k,} T represents a trailing bit (0 to 18 bits) of each pixel group.

Finally, the code generation unit 18 compares the size size (cB _k ) of the generated encoded block cB _k with the size size (B _k ) (= 512 bit) of the original non-encoded block B _k ( S209), if size (cB _k ) <size (B _k ), the encoded block cB _k is output as the compressed block B ′ _k (S210), and if size (cB _k ) ≧ size (B _k ), uncoded The block B _k is output as the compressed block B ′ _k (S211). This is because there are cases where the size of the rather compressed block B _'k by the encoding becomes larger than the previous encoding, such a case is a process for preventing an increase in data size. At the time of output, an identification flag for determining whether the compressed block B ′ _k is an encoded block or a non-encoded block is added.

(4.3) Another example of block data compression In FIG. 7B, before selecting the prediction mode PM, a method of performing SFL encoding (S205) of each residual for each prediction mode. As shown, since the SFL encoding of the residual has a high calculation complexity, this method has a high calculation cost. Therefore, as another method, a method of performing SFL encoding of the residual after selecting the prediction mode PM will be described. FIG. 9 is a flowchart illustrating another example of block data compression. In FIG. 9, the same processing parts as those in FIG. 7B are denoted by the same reference numerals. In the method of FIG. 9, since the SFL encoding of the residual (S205) is performed after the selection of the prediction mode PM (S207), before the selection of the prediction mode PM (S207), in each prediction mode PModei The block cost BC _i needs to be estimated (S206 ′).

From equation (4), ‖F‖ = 8 and ‖PM‖ = 2 are constants. CM and T in each S group are determined by the maximum value and the minimum value of the residual in the group. Therefore, first, CM and T in each S group are calculated. Since the maximum value and the minimum value are obtained and CM and T are simply determined from the values by referring to the table, the calculation cost is small. Next, for each of the four L groups (and one M group) in the block, the RGM is determined from the CM value of the S group in the L group (M group). Thereby, ‖RGM‖ is determined. At this time, since the determining group number N _g of the block, ‖CM‖, also determines ‖T‖. Finally, the code amount ‖D (S-group) ‖ of the residual of each S group is determined from the number of pixels and CM of each S group (the equation in parentheses below Equation (4)), and the sum ‖D‖ is determined. Then, the block cost BC _i can be estimated by the summation of Equation (4).

(4) Data Expansion Operation FIG. 10 is a flowchart showing the frame expansion operation by the decompressor 5 of the image processing apparatus of this embodiment. When a frame data read signal is input from the codec core 1 to the decompressor 5, the DRAM controller 7 reads the compressed frame data from the DRAM 2 and stores it in the cache memory 6 (S300). Next, the decompressor 5 performs the following processes of S302 to S308 on the compression block B ′ _k of each pixel block B _k (k = 0, 1,..., N _B −1) of 8 × 8 pixels in the frame. Execute (S301).

The prediction value decoding unit 33 determines whether or not the compressed block B ′ _k is an encoded block (S302). If the compressed block B ′ _k is an encoded block, the encoded block is decoded by the following processes of S303 to S306 to generate a pixel block B. Restore _k . On the other hand, when the compressed block B ′ _k is an uncoded block, the read compressed block B ′ _k is used as it is as the pixel block B _k (S306).

In the case where the compressed block B ′ _k is an encoded block, first, the residual decoding unit 34 _converts the data of the compressed block B ′ _k (FIG. 8) to the pixel value F of the upper left corner pixel, the prediction mode PM, and the regroup mode. RGM, encoding mode CM, encoded residual data D, and trailing bit T are separated. And based on the coding mode CM and the trailing bits T, decode all residual value in the encoding residual data D from the pixel block B _k (S303). We first determine the division of the pixel groups in regrouping mode RGM from the pixel block B _k, then stored in Table 2 (grouping table 31 from the encoding mode CM for each pixel group ) Is determined, and the encoded sequence D is decoded into the original residual value based on the encoding rule and the trailing bit T.

Next, for each pixel p _i, the predicted value decoding unit 33 first pixel block B _k from the encoded data (FIG. 8), and obtains the pixel value F and the prediction mode PM in the upper left corner pixel, the prediction mode the predicted pattern of prediction mode PM stored in the table 30 (FIG. 1), calculates the predicted value of each pixel _{p i} (S304). The frame recovery unit 35, in addition to residual values to restore the calculated pixel p _i the pixel value to the predicted values of the pixels p _i (S305). Thereby, the pixel block _Bk is restored from the encoded block.

After the processing of above S302 to S306, the output unit 36 outputs the pixel block _{B k} restored to the codec core 1 (S307).

[3] Specific Example of Hardware Mounting Next, an example in which the compressor 4 and the decompressor 5 of the present embodiment are specifically mounted on hardware will be described.

(1) Compressor FIG. 11 is a diagram illustrating a hardware implementation example of the compressor 4. In FIG. 11, the same reference numerals are given to the components corresponding to the components in FIG. The compressor 4 shown in FIG. 11 is a hardware implementation of the flow of FIG. 9 that is processed in parallel. The pixel block encoding operation process is divided into three stages, STAGE 1 to STAGE 3, and each stage is executed in parallel. By doing so, pipeline processing is possible. In STAGE 1, calculation processing of the encoding mode CM and trailing bit T of each S group in each prediction mode is executed. In STAGE 2, regrouping and prediction mode determination are performed. In STAGE 3, regrouping mode determination, residual calculation, SFL encoding and formatting are executed.

  First, the frame data output from the codec core 1 is temporarily stored in Buffer0 (frame buffer 14a). Then, it is sequentially read out every 8 × 8 pixel block, and the pixel block is temporarily stored in Buffer 1 (block buffer 15a).

  In STAGE 1, prediction calculation scanning (DPCM scanning or average prediction calculation scanning) is performed by the MDA prediction unit 16 for each M group in each prediction mode, and determination of the coding mode of each S group included in the M group is performed. Is done. As shown in FIG. 1, since PMode0 to PMode3 include a total of 36 M groups, the number of pixels to be subjected to prediction calculation scanning is 7 × 36 = 252 pixels. However, the same M group is duplicated in each prediction mode. include. Since the overlapping M group prediction calculation scans need only be performed once, the number of pixels to be actually subjected to the prediction calculation scan is 7 × 24 = 168 pixels. Among them, there are 18 groups for DPCM scanning and 6 groups for prediction calculation scanning, so in FIG. 11, three DPCM computing units (DPCM) and one average prediction computing unit (Avg) are provided in parallel. Yes. DPCM and Avg output a residual value obtained by pixel value prediction for each pixel of each M group in each prediction mode. The encoding mode detector CMD obtains the maximum value and the minimum value of the residuals output by the DPCM or Avg for each S group in each M group, and determines each of the prediction modes in each prediction mode according to Equation (3) and Table 2. The encoding mode of the S group is detected and output to the preceding register CM, and the trailing bit T is output to the preceding register Te. Also, for pipeline processing, the pixel value of the original pixel block is transferred from Buffer 1 to the previous register Original.

  In STAGE 2, first, the regrouping determination unit RGM and the three regrouping determination units RGL perform determination of the regrouping mode of each L group or M group in each prediction mode in parallel. Then, the prediction mode determiner (prediction mode selection unit 17) estimates the block cost of each prediction mode by the above-described equation (4), and selects the prediction mode with the lowest block cost. The selection prediction mode is output to the register PM at the next stage. In addition, the encoding mode CM and trailing bit T of each S group in the selected prediction mode are output to the subsequent registers CM and Te, respectively. Further, for pipeline processing, the pixel value of the original pixel block is transferred from the previous register Original to the subsequent register Original.

  In STAGE 3, the regrouping of each S group and the SFL encoding in the selected prediction mode are executed in parallel. The regrouping mode detector RGMD reads the encoding mode CM of each S group in each L or M group from the register CM at the subsequent stage, and determines each L according to FIG. 3 based on the relationship between these encoding modes CM. Alternatively, the M group regrouping mode RGM is determined. For example, if the coding mode CMs of all S groups in the L group have the same relationship, the regrouping mode RGM of the L group is determined to be RGM0. Also, if only the coding mode CM of the first S group of the L group is different from the coding modes CM of the other three S groups, the regrouping mode RGM of the L group is determined as RGM1. Is done.

  On the other hand, the residual calculator RC (3RC, 4RC, 3 / 4RC in FIG. 11) performs the operation for each S group from the pixel value stored in the subsequent register Original according to the selection prediction mode stored in the subsequent register PM. The residual from the predicted value of each pixel value is calculated. Here, 3RC represents a residual calculator of 3 pixels, 4RC represents a residual calculator of 4 pixels, and 3 / 4RC represents a residual calculator of 3 or 4 pixels. The SFL encoder (SFL encoder 25) is configured to output each S group output from each residual calculator RC based on the encoding mode and trailing bits of each S group stored in the subsequent registers CM and Te. The residual is encoded according to Table 2 and encoded residual data D is output. Finally, the code generator (code generation unit 18) includes the pixel value F at the upper left corner of the pixel block, the prediction mode PM, the regrouping mode RGM, the encoding mode CM, the encoding residual data D, and the trailing bit. T is shaped into encoded pixel block data formatted as shown in FIG. 8, the data length CDL of the encoded pixel block data is added to the head, and output to the cache memory 6.

  FIG. 12 is a diagram illustrating the timing of the pipeline processing cycle of the compressor 4 in FIG. The 18 M-group DPCM scans and the 6 M-group prediction calculation scans in STAGE 1 are executed in six cycles by three DPCM calculators (DPCM) and one average prediction calculator (Avg). Delaying one cycle from STAGE1, the regrouping of STAGE2 and the determination of the prediction mode are performed in six cycles. Determination of the regrouping mode of STAGE3, residual calculation, SFL encoding and formatting are performed in six cycles after completion of STAGE2. Therefore, the encoding process is executed in a time of 13 cycles per pixel block. Also, the encoding process for the next pixel block is continuously started from the point in time when STAGE1 for the previous pixel block is completed. Therefore, the encoding process for the next pixel block can be started from a point of delay of 6 cycles from the start time of the encoding process for the previous pixel block.

(2) Decompressor FIG. 13 is a diagram illustrating a hardware implementation example of the decompressor 5. The decompressor 5 shown in FIG. 13 is a hardware implementation of the flow of FIG. 10 that is processed in parallel. The decoding operation of the pixel block is executed in three stages, STAGE 1 to STAGE 3, and each stage is executed in parallel. By doing so, pipeline processing is possible. In STAGE 1, a process of separating the encoded data read from the DRAM 2 into F, PM, RGM, CM, and D & T parts and storing them in the preceding register is executed. In STAGE 2, after the encoded residuals are separated into M groups by a barrel shifter (BS), a process of further separating them and decoding them into individual residuals is executed. Meanwhile, trailing bits are separated from D & T and used to determine the sign of the residual after decoding. In STAGE 3, the predicted value of each pixel is calculated by DPCM prediction or average prediction, and the pixel value of each pixel is restored in addition to the residual.

The encoded pixel block read from the DRAM 2 to the cache memory 6 is data in the format shown in FIG. In STAGE 1, this data is separated. First, since the first pixel value F and the prediction mode PM are fixed sizes (respectively 8 bits and 2 bits, respectively), data of each bit length is stored in the registers F and PM in the previous stage. The subsequent regrouping mode RGM is between 13 and 17 bits long. Since the regrouping mode of the M group has a fixed length of 1 bit, the barrel shifter BS reads 1 bit as it is and stores it in the register RGM in the previous stage. On the other hand, since the regrouping mode of the L group is a variable length of 3 bits or 4 bits, the following read operations (i) and (ii) are repeatedly executed by the number of L groups (4 times).
(I) First, the barrel shifter BS reads 3 bits, and if the read bit is RG0 (“000”), 4 bits (“0000”) obtained by adding 0 to the 3 bits are stored in the preceding register RGM.
(Ii) If the read bit is other than RG0 (“000”), the barrel shifter BS reads 1 bit, and stores these 4 bits in the register RGM in the preceding stage.

  The encoding mode CM of each group is read in parallel with a delay of one cycle from the reading of the regrouping mode RGM. One encoding mode CM has a fixed length (3 bits), but the number of encoding modes CM differs depending on the regrouping mode. Therefore, after first reading out the regrouping mode RGM of one M group or L group, the IRG reads out a predetermined number of encoding modes CM according to the regrouping mode RGM. The number of bits of the read coding mode CM is 3, 6 bits for the M block and 3, 6, 9, 12 bits for the L block. For subsequent processing, the IRG assigns a coding mode CM to each S group in the M group or L group in accordance with the regrouping mode RGM in FIG. 3 and fixes 18 bits to each M group or L group. It is stored in the previous register CM as long data.

  After reading out the encoding mode CM, the encoding residual data D and the trailing bit T are read out. Since the length of the encoded pixel block is specified by the head data length CDL (Cdata length), the length of the data to be read is determined. The read data is stored in the previous register D & T.

In STAGE 2, a process of decoding the encoded residual data D and restoring the residual data is performed. Of the data stored in the preceding register D & T, the total number of bits of the encoded residual data D can be calculated from the encoding mode CM of each S group stored in the preceding register CM. Therefore, the address shifted by the total number of bits of the encoded residual data D is the leading position of the trailing bit T. The barrel shifter BS on the left side of STAGE 2 in FIG. 13 sequentially reads the encoding residual data D of each S group according to the bit length of the encoding residual data D determined from the encoding mode CM of the S group. In the encoded residual data D read out in each S group, there is data in which the first 1 bit is “1” and the other is “0” (that is, encoded residual data of one S group) D in the case where there is an encoding residual of −2 ^M−1 or 2 ^M−1 (M is an encoding mode CM) (see Table 2), barrel shifter BS on the right side of STAGE 2 in FIG. Reads one trailing bit T and stores it in register T. The read encoded residual data D in the S group is restored to the original residual data according to the code table of Table 2 by referring to the trailing bit T by the residual decoder SS & TC.

  (Example 1) When the encoding mode CM of the S group is M = 4 and the encoding residual data is D = {0010, 0000, 1100}, the encoding residual 1000 (8 or -8) is included in D. The trailing bit T is not read because it does not exist. In this case, the residual data is restored to r = {2, 0, −4} from Table 2.

  (Example 2) When the encoding mode CM of the S group is M = 4 and the encoding residual data is D = {1000,0000,1100}, the encoding residual 1000 (8 or -8) is included in D. Since it exists, the trailing bit T is read out. If the trailing bit T read out is T = 0, the encoding residual 1000 represents the residual 8. Therefore, each residual data of the S group is represented by r = {8, 0 from Table 2. , −4}. On the other hand, if the trailing bit T read out is T = 1, the encoding residual 1000 represents the residual −8, so that each residual data of the S group is represented by r = { Restored to -8, 0, -4}.

  (Example 3) When the encoding mode CM of the S group is M = 5 and the encoding residual data is D = {01000,00000,11000}, the encoding residual 10000 (16 or -16) is included in D. The trailing bit T is not read because it does not exist. In this case, the residual data is restored to r = {8, 0, −8} from Table 2.

  The restored residual data is stored in the subsequent registers d0 to d2. Further, in order to perform pipeline processing, the value of the register PM at the previous stage is copied to the register PM at the subsequent stage.

  In STAGE 3, the prediction value of each pixel is calculated from the prediction mode PM stored in the subsequent register PM and the pixel value F of the upper left corner pixel stored in the previous register F by the DPCM decoder (Inv.DPCM). The original pixel value is restored by adding the residuals calculated and added to the registers d0 to d2. The restored pixel values are arranged in the raster scan order by the recorder (Recording) and written to the final stage register (8 * 8 samples). When the entire pixel block is restored, the pixel block data is output to the codec core 1 from the final stage register (8 * 8 samples).

  FIG. 14 is a diagram illustrating the timing of the pipeline processing cycle of the decompressor 5 of FIG. One STAGE is executed in three cycles per pixel block, STAGE2 is executed with a delay of one cycle from STAGE1, and STAGE3 is executed with a delay of one cycle from STAGE2.

  In the first embodiment, an example in which the present invention is applied to a pixel block of 8 × 8 pixels is shown. In the second embodiment, as an example of extending the application of the present invention to a pixel block having 8 × 8 pixels or more, An example in which the present invention is applied to a pixel block of 16 × 16 pixels will be shown.

  FIG. 15 is a diagram illustrating a prediction mode and group division used in the MDA scheme for a 16 × 16 pixel block. FIGS. 15A, 15B, 15C, and 15D represent prediction modes PMode0, PMode1, PMode2, and PMode3, respectively. Compared to the case of FIG. 1, the number of pixels in one M group is increased from 7 pixels to 15 pixels and the number of L groups is increased. .

  In FIG. 15, the corner pixel in the 0th row and the 0th column is the starting pixel. The pixels in the pixel block excluding the starting pixel are partitioned into M groups, which are groups of (n−1) (n = 16) column pixels. In PMode0, the M group including the first to (n-1) th column pixels in the 0th row of the pixel block, and the first to (n-1) th column pixels in the (n-1) th column. Horizontal copy prediction for the M group consisting of, vertical copy prediction for each M group of even-numbered column pixels of the first to (n-1) th rows, and the first to (n-1) excluding the (n-1) th column. ) A prediction method of horizontal average prediction is assigned to each M group of odd-numbered columns of rows. PMode1 is a transpose of PMode0. In PMode2, vertical copy prediction is performed on the M group including the first to (n-1) th column pixels in the 0th column, and the M group including the column pixels on each of the first to (n-1) th columns is horizontal. Assign a prediction method for copy prediction. PMode3 is a transpose of PMode2.

  For the L group, in addition to the method of grouping into eight L groups as shown in FIG. 15, a method of grouping the eight L groups of FIG. . Any of them can be adopted.

  FIG. 16 is a diagram illustrating an example of grouping residuals of one prediction block of 16 × 16 pixels into S groups in PMode0. FIG. 16A shows an example of grouping into an S group of 7 pixels or 8 pixels. In this case, except for the increase in the number of pixels and the number of L groups, the process is the same as in FIG. 2B. Can be executed. FIG. 16B shows an example of grouping into S groups of 3 pixels or 4 pixels. In this case, if the number of S groups in one L group is set to four, which is the same as in the first embodiment, the number of L groups is four times that in the first embodiment. SFL encoding can be executed in the same manner as in the first embodiment. On the other hand, if the number of S groups in one L group is 8, the number of regrouping modes is 15 × 15 = 225, and an 8-bit code is necessary to represent the regrouping mode.

  In the third embodiment, another example of grouping of S groups in the case of the first embodiment (when the present invention is applied to an 8 × 8 pixel pixel block) will be described.

  FIG. 17 is a diagram illustrating another example in which the residuals of one prediction block of 8 × 8 pixels are grouped into S groups in PMode0. FIG. 17A shows the S group in the upper row of four S pixels in the first row to the seventh row among the S groups shown in FIG. The number of pixels is simply changed to 3 pixels. Therefore, the residual regrouping and residual SFL encoding methods described in the first embodiment can be applied as they are. FIGS. 17B and 17C show examples in which the S group is set across pixels of different prediction methods. In this case, as shown in FIGS. 17B and 17C, the method of taking the L group is also different from that in the first embodiment. However, regrouping and residual SFL encoding can be performed in the same manner as in the first embodiment.

1 Cordic core 2 DRAM
DESCRIPTION OF SYMBOLS 3 DRAM interface part 4 Compressor 5 Decompressor 6 Cache memory 7 DRAM controller 11 Prediction mode table 12 Residual memory 13 Residual calculation part 14 Frame input part 15 Block division part 16 MDA prediction part 17 Prediction mode selection part 18 Code generation part DESCRIPTION OF SYMBOLS 19 Residual encoding part 20 Output part 21 Grouping table 22 Small group division part 23 CM & T determination part 24 Regrouping part 25 SFL encoding part 30 Prediction mode table 31 Grouping table 32 Input part 33 Predictive value decoding part 34 Residual decoding Part 35 frame restoration part 36 output part

Claims (6)

A compressor for compressing frame data of an image frame,
Block dividing means for dividing the input frame data into pixel blocks of a predetermined size;
Consists of a combination of copy prediction of any one adjacent pixel and average prediction of any two adjacent pixels for each pixel in the pixel block excluding the starting pixel at any one corner of the pixel block A plurality of prediction modes, and a pixel value prediction means for calculating a prediction value of each pixel in the pixel block according to each prediction mode;
For each pixel in the pixel block, residual calculation means for calculating a residual obtained by subtracting the predicted value of the pixel from the pixel value of the pixel;
Residual encoding means for encoding the residual of each pixel in the pixel block;
For each prediction mode, the residual bit size of the pixel block encoded by the residual encoding means is calculated, and the encoded residual bit block size of the pixel block is minimized. A prediction mode selection means for selecting the prediction mode as the selected prediction mode;
The information of the pixel block including the pixel value of the starting pixel, the information of the selected prediction mode, and the residual information of the pixel block calculated and encoded in the selected prediction mode is formalized in a predetermined format. Code code generating means for generating encoded frame data;
Compressor with The block dividing means divides the frame data into pixel blocks of n × n size (n is an even number of 8 or more),
The pixel value predicting means uses the pixel at the corner of the 0th row and the 0th column as the starting pixel, and the pixels in the pixel block excluding the starting pixel as a group of (n−1) vertical pixels. And the prediction mode when the pixel value prediction means calculates the predicted value is at least:
(A) M group consisting of column pixels in the first to (n-1) th columns in the 0th row of the pixel block, and columns in the first to (n-1) th rows in the (n-1) th column. Horizontal copy prediction for the M group of pixels, vertical copy prediction for each M group of even-numbered columns of pixels in the first to (n-1) th rows, and the first to (n-) except for the (n-1) th column. 1) Prediction mode 0 in which a prediction method of horizontal average prediction is assigned to each M group of odd-numbered column pixels in a row.
(B) Prediction mode 1, which is a transpose of the prediction mode 0,
(C) Vertical copy prediction for the M group consisting of column pixels in the 1st to (n-1) th rows of the 0th column, and horizontal to each M group consisting of column pixels in each row of the 1st to (n-1) th columns. Prediction mode 2 for assigning a prediction method for copy prediction,
(D) Prediction mode 3, which is a transpose of the prediction mode 2,
The compressor according to claim 1, wherein the compressor includes four prediction modes. The residual encoding means includes
S group dividing means for dividing the residual of each pixel in the pixel block into S groups of a predetermined size that have the same pixel value prediction method by the pixel value prediction means and in which pixels are adjacent to each other;
For each S group, the absolute value of the maximum value and the minimum value of each residual belonging to the S group are calculated, and the encoding mode of the S group is calculated based on the absolute value of the maximum value and the absolute value of the minimum value. Encoding mode determining means for determining
For each of the prediction modes, the pixel block is partitioned into a plurality of L groups having a predetermined size and one M group having a half size, and each L group or M group is divided into the L group or M group. Regrouping is performed by combining the S groups belonging to the same encoding mode, and a regrouping mode is determined for each L group or M group according to the group pattern after regrouping Regrouping means to
SFL encoding means for encoding the residual of each group reorganized by the regrouping means based on a predetermined encoding table according to the encoding mode,
The code code generation means includes the pixel value of the origin pixel, the information of the selected prediction mode, the regrouping mode of each L group or M group, and the pixel block calculated and encoded in the selected prediction mode 3. The compressor according to claim 2, wherein the pixel block information including the residual information is formalized in a predetermined format. The S group dividing means divides the M group into S groups that are divided into two groups of (n / 2) pixels and (n / 2-1) pixels for each prediction mode. ,
The regrouping means includes
For the prediction mode 0, the odd-numbered M group is divided into M group pairs composed of two M groups, and each M group pair is defined as one L group, and the even-numbered M group is defined as two M groups. Each L group is formed by dividing each M group pair into one L group by dividing into M group pairs consisting of groups,
For prediction mode 2, the M groups in each row of the first to (n-1) th columns are divided into M group pairs composed of two adjacent M groups, and each M group pair is divided into one L group. The compressor according to claim 3, wherein the L groups are formed. A decompressor for restoring the frame data from the encoded frame data encoded by the compressor according to any one of claims 1 to 4,
Scanning means for sequentially scanning from the base pixel;
For the pixels sequentially designated by the scanning of the scanning means, the pixel value of the pixel based on the pixel value of the starting pixel included in the encoded frame data or the pixel value of the selected prediction mode or the already calculated adjacent pixel Prediction value decoding means for calculating a prediction value of
Residual decoding means for calculating a pixel value of the pixel based on the predicted value of the pixel calculated by the predicted value decoding means and residual information of the pixel included in the encoded frame data;
Decompressor with A codec core that encodes or decodes a moving image according to a predetermined codec method;
A frame memory that temporarily stores frame data used by the codec core for encoding or decoding operations, and an image processing apparatus comprising:
It is interposed between the codec core and the frame memory, compresses the frame data output from the codec core and stores the encoded frame data in the frame memory, and transmits the frame data from the codec core. A memory interface that reads out the encoded frame data from the frame memory and decodes it into the frame data when output is requested, and outputs the frame data to the codec core;
The memory interface is
The compressor according to any one of claims 1 to 4, wherein the frame data is compressed into the encoded frame data.
The decompressor according to claim 5, wherein the encoded frame data is decoded into the frame data.
An image processing apparatus comprising: