JP2010087673A - Image processing system and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To further improve the prediction accuracy of an image in AC component prediction. <P>SOLUTION: An area specifying part 31 specifies an area where processing by a sequential shift has not been performed as an unprocessed peripheral area and an area where processing has already been performed as a processed peripheral area in the peripheral area located in a periphery of a processing object. A prediction processor 32 calculates a prediction pixel value of subblocks s00-s11 produced segmenting a block S by AC component prediction using an average pixel value of the block S which becomes a processing object, a center pixel value of blocks T and L in the processed peripheral area, and an average pixel value of blocks B and R in the unprocessed peripheral area. The center pixel value is a linear prediction value using average pixel values of subblocks t10, t11, l01, and l11 in the processed peripheral area and the average pixel value of the block S, and based on an intercentral distance of the block and the subblocks. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing system and an image processing program, and more particularly to AC component prediction (ACP: AC Componet Prediction).

  Conventionally, an image processing method called AC component prediction (ACP) is known. AC component prediction refers to a method for obtaining information on a small area obtained by subdividing a pixel block by referring to information on the peripheral area of the pixel block to be processed, and using spatial correlation with the surrounding area. It is a thing. For example, Patent Documents 1 to 3 disclose recursive alternating current component predictive coding that sequentially subdivides blocks to be processed and hierarchically encodes image data. Specifically, first, the predicted pixel values of sub-blocks obtained by subdividing the block to be processed are sequentially calculated while sequentially shifting the block to be processed in a predetermined direction on the image plane. This predicted pixel value is calculated by AC component prediction with reference to the peripheral region to be processed. Next, a difference between the predicted pixel value and the original pixel value (true value) is calculated as a prediction residual. Then, an irreversible transformation and entropy coding are performed on the prediction residual, thereby generating an AC component of an image that is a part of the compressed data. The above processing is performed on 8 × 8 pixel blocks (top layer), 4 × 4 pixel blocks (second layer), 2 × 2 pixel blocks (third layer), and 1 × 1 pixel blocks. It is recursively repeated in a hierarchical structure consisting of blocks (lowest hierarchy).

  In particular, in Patent Document 1, in order to improve the prediction accuracy, among the peripheral regions that are referred to in the AC component prediction, for regions that have already been processed by the sequential shift, sub-blocks with higher correlation are provided. A technique for referring to information is disclosed. For example, when a block of 4 × 4 pixels is processed in a line-sequential scan in the horizontal direction, peripheral regions adjacent to the top and left of the processing target are processed. In this case, the processed peripheral area is not the average pixel value of the block itself but the average pixel of two sub-blocks adjacent to the processing target among the 2 × 2 pixel sub-blocks obtained by subdividing the block into four. Refers to the value. The average of the two average pixel values is assumed to be the “center pixel value” of the 8 × 8 pixel block, and the AC component prediction is performed on the assumption of this. Here, the “center pixel value” refers to the true pixel value at the center position of the pixel block at infinite resolution.

Japanese Patent No. 4000157 Japanese Patent No. 3774201 Japanese Patent No. 3700906

  As is well known, it is preferable that the image prediction accuracy by the AC component prediction is high. This is because the higher the prediction accuracy, the higher the correlation between the predicted image as a set of predicted pixel values and the original image, and the smaller the prediction residual corresponding to the difference between the two. The reduction of the prediction residual means that the tendency of the appearance frequency of the prediction residual statistically viewed tends to be close to 0. Therefore, for example, when entropy coding is performed on the prediction residual to compress the data, the compression rate can be increased by reducing the prediction residual. In this regard, the above-described method of Patent Document 1 still leaves room for improving the prediction accuracy. This is because the average pixel value of the sub-block is assumed to be the center pixel value of the large block to which it belongs without considering the difference in the center position of the multiple types of blocks. is there.

  The present invention has been made in view of these points, and an object thereof is to further improve the image prediction accuracy in the AC component prediction.

  In order to solve such a problem, the first invention sets a plurality of first blocks on the image plane, and sequentially shifts the first block to be processed in a predetermined direction while changing the first block. An image processing system that performs AC component prediction is provided. This image processing system includes an area specifying unit and a prediction processing unit. The region specifying unit specifies, as a non-processed peripheral region, a region that has not yet been sequentially processed by the shift among peripheral regions located around the processing target. In addition, the region specifying unit specifies, as a processed peripheral region, a region in which processing has already been performed among peripheral regions located around the processing target by sequential shift. The prediction processing unit uses the average pixel value of the first block to be processed, the central pixel value of the first block in the processed peripheral area, and the average pixel value of the first block in the unprocessed peripheral area. The predicted pixel value of the second block obtained by subdividing the first block to be processed is calculated by the AC component prediction. Here, the central pixel value uses the average pixel value of the second block in the processed peripheral area and the average pixel value of the first block to be processed, and the first block and the second block It is a linear prediction value based on the center-to-center distance.

  Here, in the first invention, it is preferable to use the second block positioned adjacent to the processing target as the second block in the processed peripheral area.

  In the first invention, the average pixel value of the second block is calculated by adding a prediction residual corresponding to the difference from the true pixel value to the prediction pixel value calculated by the prediction processing unit. An adder may be further provided. In this case, it is preferable that the average pixel value of the second block calculated by the adder is fed back to the prediction processing unit as information on the processed peripheral area necessary for the subsequent processing.

  In the first invention, a plurality of hierarchical processing units may be provided in which the size of a block to be processed decreases stepwise as the hierarchical level becomes lower. In this case, each of the hierarchical processing units includes the region specifying unit, the prediction processing unit, and the adder. The average pixel value of the second block calculated by the adder on the upper layer side is used as the average pixel value of the first block that is the processing target and unprocessed peripheral area on the lower layer side. Supplied to the processing unit.

  In the first invention, the image processing system may be an encoder that compresses an image. In this case, a subtractor that calculates a difference between the predicted pixel value calculated by the prediction processing unit and the true pixel value as a prediction residual, and a non-reversible conversion is performed on the prediction residual calculated by the subtractor. An irreversible transform unit, an entropy coder that generates an AC component of an image as part of compressed data by performing entropy coding on a prediction residual subjected to the irreversible transform, and an irreversible It is preferable to further provide an inverse transform unit that generates the prediction residual to be supplied to the adder by performing inverse processing of the lossy transformation on the transformed prediction residual.

  Furthermore, in the first invention, the image processing system may be a decoder that decompresses an image. In this case, an inverse transform unit that restores the prediction residual to be supplied to the adder by performing inverse processing of irreversible transformation and entropy encoding performed at the time of image compression on the compressed data of the image is further provided. It is preferable to provide it.

  On the other hand, the second invention provides an image processing program in which a plurality of first blocks are set on an image plane and AC component prediction is performed while sequentially shifting the first block to be processed in a predetermined direction. . The program includes a first step of identifying, as an unprocessed peripheral area, an area that has not yet been subjected to sequential shift processing among peripheral areas positioned around the processing target, and sequential shift of the peripheral areas. A second step of identifying an area where processing has already been performed as a processed peripheral area, an average pixel value of the first block to be processed, and a central pixel value of the first block in the processed peripheral area; A third step of calculating a predicted pixel value of a second block obtained by subdividing the first block to be processed by AC component prediction using the average pixel value of the first block in the unprocessed peripheral region; The computer is caused to execute image processing having Here, the central pixel value uses the average pixel value of the second block in the processed peripheral area and the average pixel value of the first block to be processed, and the first block and the second block It is a linear prediction value based on the center-to-center distance.

  Here, in the second invention, it is preferable to use the second block that is positioned adjacent to the processing target as the second block in the processed peripheral region.

  In the second invention, a mean pixel value of the second block is calculated by adding a prediction residual corresponding to a difference from a true pixel value to the calculated prediction pixel value. A step may be further provided. In this case, it is preferable to use the calculated average pixel value of the second block as information on the processed peripheral area in the subsequent processing.

  In the second invention, the image processing is recursively executed in a hierarchical structure in which the size of a block to be processed becomes smaller step by step as the hierarchy becomes lower, and the second value calculated on the upper hierarchy side. It is preferable to use the average pixel value of the first block as the average pixel value of the first block to be processed and the unprocessed peripheral area on the lower layer side.

  In the second invention, the image processing program may be an encoding program for compressing an image. In this case, a fifth step of calculating a difference between the calculated predicted pixel value of the second block and the true pixel value as a prediction residual, and an irreversible transformation on the calculated prediction residual And a seventh step of generating an AC component of the image as a part of the compressed data by performing entropy coding on the prediction residual subjected to the irreversible transformation. The prediction residual used for calculation of the average pixel value of the second block is generated by performing reverse processing of the lossy conversion on the prediction residual subjected to the lossy conversion. It is preferable to further provide a step.

  Furthermore, in the second invention, the image processing program may be a decoding program for expanding an image. In this case, the prediction residual used for calculating the average pixel value of the second block is obtained by performing inverse processing of lossy transformation and entropy encoding performed at the time of image compression on the compressed data of the image. It is preferable to further provide a ninth step of generation.

  In the AC component prediction, the predicted pixel value of the second block obtained by subdividing the first block to be processed is calculated using the spatial correlation of the peripheral region located around the processing target. At that time, among the peripheral areas to be referred to, for the processed peripheral areas for which the processing by the sequential shift has already been performed, the center pixel value is used instead of the average pixel value of the first block. The center pixel value is determined by taking into consideration the distance between the centers of the first and second blocks, the average pixel value of the second block in the processed peripheral area, and the average pixel value of the first block to be processed. Specified as a linear prediction value based on. Therefore, compared to the conventional method in which the average of the average pixel values of the small blocks is assumed to be the center pixel value of the large block, the center pixel value closer to the true value can be calculated, so that the image prediction accuracy is further improved. Can be achieved.

  Hereinafter, as an embodiment to which the AC component prediction according to the present invention is applied, recursive AC component prediction coding (RACP: Recursive ACP) will be described as an example.

(RACP encoder)
FIG. 1 is an overall configuration diagram of a RACP encoder. This encoder has a DC calculation unit 1, a DC encoding unit 2, and hierarchical processing units 3a to 3c, and is configured by four layers in this embodiment. Data TDCn, DCn (n = 0, 1, 2, 3) output from these units 1, 2, 3a to 3c are temporarily stored in a buffer (storage unit) (not shown).

  The DC calculation unit 1 divides an input image to be compressed into blocks having a preset size, and outputs the average pixel value of each block, that is, the average value of pixels included in the block, as TDC0 to TDC3. . Here, TDC0 is an average pixel value of a block of 8 × 8 pixels (hereinafter referred to as “8 × 8 block”), TDC1 is an average pixel value of a block of 4 × 4 pixels (hereinafter referred to as “4 × 4 block”), and TDC2 Is an average pixel value of a 2 × 2 pixel block (hereinafter referred to as “2 × 2 block”). TDC3 is an average pixel value of a 1 × 1 pixel block (hereinafter referred to as “1 × 1 block”), that is, a pixel value itself of a pixel which is a minimum unit of an image. The average pixel values TDC0 to TDC3 (true values) calculated directly from the input image are distinguished from the average pixel values DC0 to DC3 (restored values) restored through the processing in the units 2 and 3a to 3c. Please note that Since both values are accompanied by irreversible transformation in the process of encoding, they do not exactly match.

  The DC encoding unit 2 and the hierarchical processing units 3a to 3c include 8 × 8 blocks (top layer), 4 × 4 blocks (second layer), 2 × 2 blocks (third layer), and 1 × 1 block. In the four-layer structure composed of (the lowest layer), the layer processing assigned to itself is performed.

  The DC encoding unit 2 in the highest layer performs differential pulse code modulation (DPCM) and entropy encoding on the average pixel value TDC0 for each 8 × 8 block read from the buffer. Difference pulse code modulation encodes the difference between the average pixel values TDC0 for blocks adjacent to each other on the image plane. Then, entropy coding such as Huffman coding or arithmetic coding is performed on the coded difference after quantization. The data that has undergone such processing is output as a DC component DC0 of an image that is a part of the compressed data, and an average pixel value of 8 × 8 blocks is supplied to the hierarchical processing unit 3a of the immediately lower hierarchy. A restored (inverse quantized) value DC0 is output.

  The three hierarchical processing units 3a to 3c include an average pixel value TDCn (n = 1, 2, 3, and so on) generated by the DC calculation unit 1 and an average pixel value DCn−1 generated by the immediately higher hierarchy. As a part of the compressed data, the AC component ACn of the image is output by the process including the AC component prediction. At the same time, the hierarchical processing units 3a to 3c restore the average pixel value DCn from the AC component ACn. The restored average pixel value DCn is output to feed back to the same hierarchy and to supply it to the immediately lower hierarchy if necessary.

  Specifically, the hierarchical processing unit 3a in the second hierarchical level undergoes processing such as AC component prediction and prediction residual calculation for 8 × 8 blocks, and 4 × 4 blocks of AC components AC1 and 4 The average pixel value DC1 of x4 blocks is output. In this hierarchy, the average pixel value DC0 of the 8 × 8 block generated in the highest hierarchy and the average pixel value DC1 of the 4 × 4 block fed back in the same hierarchy are used as information to be referred to in the AC component prediction. It is done. At the same time, an average pixel value TDC1 of 4 × 4 blocks calculated by the DC calculation unit 1 is input to calculate a prediction residual. The hierarchical processing unit 3b of the third hierarchical level performs processing such as AC component prediction on 4 × 4 blocks as processing targets, and outputs 2 × 2 blocks of AC components AC2 and 2 × 2 blocks of average pixel values DC2. Output. In this hierarchy, the average pixel value DC1 of 4 × 4 blocks generated in the second hierarchy and the average pixel value DC2 of 2 × 2 blocks fed back in the same hierarchy are used as reference information for AC component prediction. . At the same time, an average pixel value TDC2 of 2 × 2 blocks calculated by the DC calculation unit 1 is input to calculate a prediction residual. The lowermost hierarchy processing unit 3c outputs an AC component AC3 of 1 × 1 block and an average pixel value DC3 of 1 × 1 block through processing such as AC component prediction for 2 × 2 blocks. To do. In this hierarchy, the 2 × 2 block average pixel value DC2 generated in the third hierarchy and the 1 × 1 block average pixel value DC3 fed back in the same hierarchy are used as reference information for AC component prediction. . At the same time, the average pixel value TDC3 of 1 × 1 block calculated by the DC calculation unit 1 is used to calculate the prediction residual.

  In this way, in a multi-hierarchical structure in which the sizes of blocks to be processed are different, the DC encoding unit 2 and the hierarchical processing units 3a to 3c are linked to each other so that image processing mainly based on AC component prediction is recursively performed. Executed. As a result, the DC component DC0 of the image and the AC components AC1 to AC3 are output as compressed data. The compressed data includes accompanying information such as a Huffman table in addition to these.

  FIG. 2 is an explanatory diagram of blocks set on the image plane. A plurality of blocks are set on the image plane by dividing an image to be compressed (one frame image or a partial image thereof) vertically and horizontally, and processed in units of blocks. The block size is set to decrease step by step as the hierarchy becomes lower, and the size of the block to be processed in a certain hierarchy is the size of the sub-block subdivided in the immediately higher hierarchy Matches. Processing of the entire image in a certain hierarchy is achieved by repeating the processing while sequentially shifting the blocks to be processed on the screen and processing all the blocks in the image. As shown in the figure, the direction of this shift may be a line-sequential scanning along the horizontal direction, but may be arbitrarily set including that along the vertical direction. Further, the shift directions in the respective hierarchies are not necessarily the same. Here, assuming that the block to be processed is S, the block S is obtained by the AC component prediction with reference to the peripheral regions located in the periphery of the block S, in this embodiment, the blocks T, B, L, and R adjacent in the vertical and horizontal directions. The predicted pixel value of each sub-block obtained by subdividing is calculated. In the present specification, among the peripheral regions T, B, L, and R, blocks B and R that have not yet undergone sequential shift processing are defined as “unprocessed peripheral regions”. In the blocks B and R corresponding to the unprocessed peripheral area, the average pixel value (predicted pixel value) of the sub-blocks obtained by subdividing the blocks has not yet been calculated, and the entire blocks B and R calculated in the immediately higher hierarchy Only the average pixel value is known. In addition, among the peripheral areas T, B, L, and R, blocks T and L that have already been processed by sequential shift are defined as “processed peripheral areas”. In the blocks T and L corresponding to the processed peripheral area, the average pixel value (predicted pixel value) of the sub-block obtained by dividing the block into four is calculated by the previous process in the same hierarchy. Hereinafter, sub-blocks obtained by subdividing a block are identified by subscripts “00” in the upper left, “01” in the upper right, “11” in the lower right, and “10” in the lower left (see FIG. 9). ).

  In the present embodiment, the upper, lower, left, and right T, B, L, and R of the block S to be processed are set as the peripheral area. It is good also as an area | region, and you may include other than that (for example, diagonal direction, one jump etc.) in a peripheral area.

  The processing of the RACP encoder may be either parallel processing or sequential processing. FIG. 3 is an operation timing chart in the parallel processing. First, the DC calculation unit 1 operates to generate average pixel values TDC0 to TDC3. Then, as the generation of all the average pixel values TDC0 to TDC3 is completed, the DC encoding unit 2 and the hierarchical processing units 3a to 3c operate in parallel. However, the timing at which these operations start is not the same, and the lower the hierarchy, the later the start timing. This delay is caused by processing delay due to sequential shift of the upper layer. Referring to FIG. 2, when processing a block S at a certain level, the processing of the peripheral regions T, B, L, and R positioned at the top, bottom, left, and right ends in the level immediately above (conditional processing Start condition). In other words, the processing of the block S in the lower layer cannot be started unless the sequential shift proceeds and the processing of the block B is finished in the upper layer. This is the reason why the operation delay occurs in the lower layer. However, compared with the sequential processing as shown in FIG. 4, the pipelined parallel processing shown in FIG. 3 is faster. In the sequential processing of FIG. 4, the processing of the immediately lower layer is started when all processing of a certain layer is completed. Therefore, the hierarchical processing start condition is naturally satisfied even in the sequential processing.

  FIG. 5 is a configuration diagram of the hierarchy processing unit 3 (generic name for 3a to 3c) in the RACP encoder. The hierarchy processing unit 3 in each hierarchy has the same configuration and process flow except that the handled block sizes are different. The hierarchical processing unit 3 includes a region specifying unit 31, a prediction processing unit 32, a subtractor 33, an irreversible conversion unit 34, an entropy encoding unit 35, an inverse conversion unit 36, and an adder 37.

  The area specifying unit 31 specifies an unprocessed peripheral area and a processed peripheral area related to the current processing target. As shown in FIG. 2, when the block to be processed is S, the lower block B and the right block R are unprocessed peripheral areas, and the upper block T and the left block L are processed peripheral areas. These areas are discriminated (processed / unprocessed) by the size of the block to be processed (for example, 8 × 8 pixels), the size of the screen (for example, 1024 × 768 pixels), and the shift direction (for example, along the horizontal line). Assuming that (line sequential scanning) is already known, it is uniquely identified from the number of the block currently being processed (hereinafter referred to as “block number BN”). For the blocks S, B, and R that are the processing target and the unprocessed peripheral area, the average pixel value DCn−1 of the block (for example, 8 × 8 block) is read from the buffer. These average pixel values DCn−1 have already been calculated in the immediately higher hierarchy and are supplied from the immediately higher hierarchy via a buffer. On the other hand, for the blocks T and L that are processed peripheral areas, the average pixel value DCn of sub-blocks t and l (for example, 4 × 4 blocks) obtained by subdividing the blocks is read from the buffer. These average pixel values DCn have already been calculated by the previous process of sequential shift in the same hierarchy, and are fed back through the buffer. It is not necessary to read out the average pixel value DCn of all the sub-blocks t and l, and the one adjacent to the block S is read out. Specifically, the average pixel value DCn of the lower two sub-blocks t10 and t11 adjacent to the block S among the four sub-blocks t constituting the upper block T. Due to the spatial correlation of images, these sub-blocks t10 and t11 tend to reflect the characteristics of the block S darker than the upper block T. Therefore, when the AC component of the block S is predicted, it is advantageous to increase the prediction accuracy by referring to the main parts t10 and t11 instead of the entire upper block T. Similarly, the left block L is the average pixel value DCn of the right two sub-blocks l01 and l11 adjacent to the block S among the four subblocks l constituting the left block L. Then, the read average pixel values DCn−1 and DCn are supplied to the prediction processing unit 32.

  The prediction processing unit 3 calculates the average pixel value DCn-1 of the block S to be processed, the center pixel values CL and CT of the blocks T and L in the processed peripheral area, and the average of the blocks R and B in the unprocessed peripheral area. Predicted pixel values of sub-blocks s00 to s11 obtained by subdividing the block S are calculated by AC component prediction using the pixel value DCn-1. As shown in FIG. 9, the center pixel value CL (or CT) uses the average pixel value DCn of the sub-blocks l01 and l11 (or t10 and t11) and the average pixel value DCn-1 of the block S.

  The center pixel values CL and CT correspond to linear prediction values in consideration of the distance between centers of a plurality of types of blocks (for example, 8 × 8 blocks and 4 × 4 blocks). 6 is an explanatory diagram of the distance between the centers of the blocks focusing on the left block L, and FIG. 7 is an explanatory diagram of calculating the central pixel value CL by linear prediction. The central pixel value CS at the center position of the block S is assumed to be its own average pixel value DCn-1. Further, the stop pixel value Cl01 at the center position of the sub-block l01 is assumed to be its own average pixel value DCn, and the stop pixel value Cl11 at the center position of the sub-block 111 is assumed to be its own average pixel value DCn. Here, if the length of one side of the square when the sub-blocks l01 and 11 are further divided into four is 1, the distance between the centers of the sub-blocks l01 and l11 is 2. Further, if the pixel value at the midpoint that divides the distance between the sub-blocks l01 and l11 into 1: 1 is p0, p0 is (Cl01 + Cl11) / 2. Since this intermediate point divides the distance between the centers of the blocks S and L into 1: 3, the central pixel value CL of the block L is (4p0−CS) / 3. In the above calculation method, the pixel value p1 at the midpoint on the extension line of the straight line connecting the center position of the block S and the center position of the sub-block l01 and the extension of the straight line connecting the center position of the block S and the sub-block l11. This is equivalent to calculating the pixel value p2 of the intermediate point on the line and taking the average of these pixel values p1 and p2. According to this method, the center pixel value CL closer to the true value is compared with the conventional method as shown in FIG. 8, that is, the method in which the intermediate pixel value p0 is treated as it is as the center pixel value CL of the block L. Can be calculated. The above description is the same for the upper block S.

  Referring to FIG. 5 again, the prediction processing unit 32 includes a center value prediction unit 32a and an AC component prediction unit 32b. The center value prediction unit 32a calculates the center pixel values T and L of the upper and left blocks based on the following Equation 1. Note that S, T, L, t10, t11, l01, and l11 do not indicate blocks and sub-blocks themselves, but indicate their center pixel values as long as they are expressed in mathematical formulas.

(Formula 1)
T = (2 × (t10 + t11) −S) / 3
L = (2 × (l01 + l11) −S) / 3

  Further, the AC component prediction unit 32b calculates a predicted pixel value C (= c00 to c11) of sub-blocks s00 to s11 obtained by subdividing the block S to be processed, that is, a predicted image, based on the following formula 2. . As far as the expression 2 is concerned, S, T, B, L, and R do not indicate the block itself, but S, R, and B indicate the average pixel value, and T and L indicate the above-described central pixel value. And The calculated predicted pixel value C is supplied to the subtracter 33 and the adder 37.

(Formula 2)
c00 = S + (T−B + LR) / 8
c01 = S + (T−B−L + R) / 8
c10 = S + (-T + B + LR) / 8
c11 = S + (− T + B−L + R) / 8

  When a part of the peripheral area is missing like an edge on the image plane, the average pixel value DCn-1 of the block S to be processed is exceptionally used as information on that part. . Further, in order to reduce the processing load, it is also possible to integrate the formulas 1 and 2 to calculate the predicted pixel value C by one calculation.

  The subtractor 33 is based on the predicted pixel value C (for example, 4 × 4 block unit) calculated by the prediction processing unit 32a and the corresponding average pixel value TDCn (for example, 4 × 4 block unit), that is, based on the input image. The difference from the true pixel value calculated in this way is calculated as the prediction residual PR. As shown in FIG. 9, the closer the predicted image (a set of predicted pixel values c00 to c11) generated by the AC component prediction is to the original image (true pixel value corresponding to the predicted pixel values c00 to c11), The prediction residual PR is reduced. The smaller the prediction residual PR, the more likely the appearance frequency of the prediction residual statistically tends to be close to 0, which is advantageous in performing entropy coding.

  The irreversible transform unit 34 performs irreversible transforms such as Hadamard transform and quantization on the prediction residual PR calculated by the subtractor 33. The entropy coding unit 35 performs entropy coding such as Huffman coding or arithmetic coding on the prediction residual PR subjected to the irreversible transformation, thereby, for example, alternating current component ACn (4 × 4 block unit ACn ( Generate compressed data)). The inverse transform unit 36 calculates a prediction residual PR ′ by performing reverse processing of the lossy transformation on the prediction residual PR subjected to the lossy transformation. This prediction residual PR ′ is a restoration of the original prediction residual PR, but is an irreversible transformation (the original value cannot be completely restored)). It is slightly different from PR. The adder 37 calculates the average pixel value DCn of the sub-block (for example, 4 × 4 block) by adding the prediction residual PR ′ to the prediction pixel value C calculated by the prediction processing unit 32. The reason why the restored prediction residual PR ′ is used as the input of the adder 37 instead of the original prediction residual PR is that accumulation of errors is prevented by iterative processing during decoding, and the reproducibility of the expanded image is ensured. It is to do. The average pixel value DCn output from the adder 37 is temporarily stored in the buffer as information on the processed peripheral area required for subsequent processing in the sequential shift. The average pixel value DCn stored in the buffer is appropriately read out by the region specifying unit 31 and fed back to the prediction processing unit 32 in the same hierarchy. At the same time, the average pixel value of the block to be processed and the unprocessed peripheral area (for example, 4 × 4 blocks) in the immediately lower hierarchy is also supplied to the prediction processing unit 32 in the immediately lower hierarchy.

  Thus, according to the RACP encoder according to the present embodiment, the average pixel values of the sub-blocks t10, t11, l01, and l11 in the processed peripheral region are used as the central pixel values of the blocks T and L in the processed peripheral region. A linear prediction value using the average pixel value of the block S to be processed is used. In calculating the linear prediction value, the distance between the centers of the blocks and sub-blocks is taken into consideration. Therefore, since the value with higher reliability than the conventional technique can be calculated as the central pixel value of the blocks T and L in the processed peripheral region, the image prediction accuracy can be further improved. As a result, the prediction residual PR, which is a difference from the true pixel value, can be reduced, and further improvement in compression efficiency can be expected. In particular, if pipelined parallel processing as shown in FIG. 3 is performed, overall processing time can be reduced.

(RACP encoding program)
Next, a description will be given of a RACP encoding program for realizing processing equivalent to a RACP encoder realized as hardware by software processing of a computer. Since there is no essential difference between hardware processing and software processing, only a brief description will be given here, and the details will be referred to the above description.

  FIG. 10 is a flowchart of the RACP encoding program. In software processing by a computer, sequential processing as shown in FIG. 4 is fundamental. First, in step 1, a plurality of blocks (4 types) are set on the image plane of the input image, and average pixel values TDC0 to TDC3 of each block are calculated. The calculated average pixel values TDC0 to TDC3 are stored in the buffer as needed. When all the blocks have been processed, the process proceeds to Step 2.

  In Step 2, DC coding, that is, differential pulse width modulation and entropy coding is performed on the average pixel value TDC0 of the 8 × 8 block, and thereby the DC component of the image that becomes part of the compressed data. DC0 is generated and output. The restored value is stored in the buffer as DC0.

  In step 3, the hierarchy number LN is set to 1 (initial value). LN = 1 indicates that the hierarchy to be processed is the highest hierarchy. Each time one hierarchical process is completed, it is incremented by 1 (step 14), and the process proceeds in order to the lower hierarchy. Then, when LN = 3, that is, when the processing of the lowest layer is finished, all the processing is finished (step 13).

  In step 4, the block number BN is set to 1 (initial value). With the setting of the layer number LN in the previous step 3, the size of the block S to be processed in that layer is uniquely specified, and the end block number BNend corresponding to the total number of blocks is also specified. Each time processing of a block in the same hierarchy is completed, the block number BN is incremented by 1 (step 12), and the processing target is sequentially shifted in a predetermined direction. Then, when the processing of the block corresponding to the end block number BNend is finished, the processing in the hierarchy is finished (step 11).

  In step 5, with respect to the block S designated by the block number BN, the center pixel values T and L of the upper block and the left block in the processed peripheral region are calculated by the linear prediction based on the above-described Expression 1.

  In step 6, the AC component prediction is performed on the block S specified by the block number BN based on the above-described equation 2, and thereby, the predicted pixel values C (= c00) of the sub-blocks s00 to s11 obtained by subdividing the block S are obtained. ˜c11) is calculated. In step 7 following this, the difference between the predicted pixel value C calculated in the previous step 6 and the true pixel value TDCn is calculated as the prediction residual PR.

  In step 8, irreversible transformation and entropy coding are performed on the prediction residual PR calculated in the previous step 7. As a result, an AC component ACn of the image as a part of the compressed data is generated. In subsequent step 9, the prediction error PR subjected to the irreversible transformation is subjected to the inverse process to calculate a prediction error PR ′ (restored value) obtained by restoring the original prediction error PR. In step 10, by adding the prediction residual PR ′ to the prediction pixel value C calculated in step 6, the average pixel value DCn of the sub-block is calculated and stored in the buffer.

  In step 11, it is determined whether or not the block number BN has reached the end block number BNend. Until the end block number BNend is reached, the block number BN is incremented by 1 in step 12, and the processing in steps 5 to 11 is repeated. When the end block number BNend is reached, that is, when the processing of all the blocks in the image plane is completed, the process exits the loop and proceeds to step 13.

In step 13, it is determined whether or not the hierarchy number LN has reached 3 (the lowest hierarchy). Until it reaches 3, the layer number LN is incremented by 1 in step 14, and the loop of steps 5 to 12 is repeated. As a result, the size of the block S to be processed gradually decreases as the hierarchy becomes lower, and the series of processes described above are recursively executed. The average pixel value DCn (output) of the sub-block calculated on the upper layer side is used as the average pixel value DCn-1 (input) of the block to be processed on the lower layer side and the unprocessed peripheral region. Then, when the block number BN reaches 3, that is, when the processing of the lowest layer is completed, the loop exits from the determination result of step 11, thereby ending the processing of this program.

  According to the RACP encoding program according to the present embodiment, as with the above-described RACP encoder, it is possible to further improve the prediction accuracy of an image in AC component prediction, and to expect an improvement in compression efficiency.

(RACP decoder)
FIG. 11 is an overall configuration diagram of a RACP decoder that decompresses compressed data generated by the above-described RACP encoder or RACP encoding program. This decoder is mainly composed of DC decoding 5 and three hierarchical processing units 6a to 6c. Data DCn (n = 0, 1, 2, 3) output from these units 5, 6a to 6c is temporarily stored in a buffer (storage unit) (not shown).

  The DC decoding 5 and the hierarchy processing units 6a to 6c are configured to have 8 × 8 blocks (the highest layer), 4 × 4 blocks (the second layer), 2 × 2 blocks (the third layer), and 1 × 1 block ( In the hierarchical structure of the lowest hierarchy, the hierarchical processing assigned to itself is performed. The DC decoding unit 5 in the highest hierarchy generates an average pixel value DC0 of 8 × 8 blocks by performing reverse processing of the processing applied at the time of image compression on the compressed data related to the DC component DC0 of the image. This is supplied to the hierarchy processing unit 6a of the second hierarchy.

  The hierarchical structure itself of the RACP decoder is substantially the same as that of the RACP encoder, but the AC components AC1 to AC3 that are the outputs of the hierarchical processing units 3a to 3c in the encoder are input to the hierarchical processing units 6a to 6c in the decoder. Is different. These hierarchical processing units 6a to 6c restore the average pixel value DCn based on the compressed data related to the AC components AC1 to AC3 of the image and the average pixel value DCn-1 supplied from the immediately higher hierarchy. The restored average pixel value DCn is output to feed back to the same hierarchy and to be supplied to the immediately lower hierarchy if necessary, and is stored in the buffer. Then, a set of average pixel values DC3 of 1 × 1 blocks calculated by the lowest-level hierarchy processing unit 6c is the final decompressed image. Note that the direction of sequential shift in the decoder conforms to that of the encoder. The decoder processing may be either parallel processing or sequential processing.

  FIG. 12 is a configuration diagram of the hierarchical processing unit 6 (generic name of 6a to 6c) in the RACP decoder. The hierarchy processing unit 6 in each hierarchy has the same basic configuration and operation except that the size of blocks to be handled is different. The hierarchy processing unit 6 includes a region specifying unit 61, a prediction processing unit 62, an adder 63, and an inverse conversion unit 64.

The area specifying unit 61 specifies an unprocessed peripheral area and a processed peripheral area related to the current processing target by the same method as the RACP encoder. That is, as shown in FIG. 2, for the blocks S, B, and R to be processed and unprocessed peripheral areas, the average pixel value DCn-1 of the block itself (for example, 8 × 8 blocks) is read from the buffer. . These average pixel values DCn−1 have already been calculated in the immediately higher hierarchy and are supplied from the immediately higher hierarchy via a buffer. On the other hand, for the blocks T and L that are processed peripheral areas, the average pixel value DCn of sub-blocks t and l (for example, 4 × 4 blocks) obtained by subdividing the blocks is read from the buffer. These average pixel values DCn have already been calculated by the previous processing in the shift order in the same hierarchy, and are fed back through the buffer. The data that needs to be read is the average pixel value DCn of the lower two sub-blocks t10 and t11 adjacent to the block S with respect to the four sub-blocks t constituting the upper block T. For one sub-block l, the average pixel value DCn of the right two sub-blocks l01 and l11 adjacent to the block S is shown. Then, the read average pixel values DCn−1 and DCn are supplied to the prediction processing unit 62.
The prediction processing unit 62 calculates the average pixel value DCn−1 of the block S to be processed, the center pixel values CL and CT of the blocks T and L in the processed peripheral area, and the average of the blocks R and B in the unprocessed peripheral area. Predicted pixel values of sub-blocks s00 to s11 obtained by subdividing the block S are calculated by AC component prediction using the pixel value DCn-1. As shown in FIG. 13, the center pixel value CL (or CT) uses the average pixel value DCn of the sub-blocks l01 and l11 (or t10 and t11) and the average pixel value DCn-1 of the block S, and This is a linear prediction value considering the distance between centers of a plurality of types of blocks (for example, 8 × 8 blocks and 4 × 4 blocks).

Specifically, the prediction processing unit 62 includes a center value prediction unit 62a and an AC component prediction unit 62b. The center value prediction unit 62a calculates the center pixel values T and L of the upper and left blocks based on the above-described formula 1. Further, the AC component prediction unit 62b calculates the predicted pixel values C (= c00 to c11) of the sub-blocks s00 to s11 obtained by subdividing the block S to be processed, that is, the predicted image, based on the above-described Expression 2. . The calculated predicted pixel value C is supplied to the adder 63. When a part of the peripheral area is missing like an edge on the image plane, it is processed according to a rule decided in advance with the decoder. For example, the average pixel value DCn-1 of the block S to be processed is supplemented as the information of that portion.

  The inverse transform unit 64 should supply the adder 63 by performing inverse processing of lossy transformation and entropy encoding performed at the time of image compression on the compressed data of the AC component ACn of 4 × 4 blocks, for example. The prediction residual PR ′ is restored. The adder 63 adds the prediction residual PR ′ reconstructed by the inverse transform unit 64 to the predicted pixel value C calculated by the prediction processing unit 62 to thereby obtain an average pixel of the sub-block (for example, 4 × 4 block). The value DCn is calculated (see FIG. 13). The average pixel value DCn output from the adder 63 is temporarily stored in the buffer as information on the processed peripheral area necessary for subsequent processing in the sequential shift. The average pixel value DCn stored in the buffer is appropriately read out by the region specifying unit 61 and fed back to the prediction processing unit 62 in the same hierarchy. At the same time, the average pixel value of the block to be processed and the unprocessed peripheral area (for example, 4 × 4 blocks) in the immediately lower hierarchy is also supplied to the prediction processing unit 62 in the immediately lower hierarchy.

  As described above, according to the RACP decoder according to the present embodiment, the compressed data generated by the above-described RACP decoder or RACP encoding program can be appropriately decompressed. At the same time, as the central pixel value of the blocks T and L in the processed peripheral area, a value with higher reliability than that of the conventional technique can be calculated, so that it is possible to expand the image taking advantage of high prediction accuracy. In particular, if pipelined parallel processing as shown in FIG. 3 is performed, overall processing time can be reduced.

(RACP decoding program)
Next, a RACP decoding program for realizing processing equivalent to the RACP decoder realized as hardware by software processing of a computer will be described. Since there is no essential difference between hardware processing and software processing, only a brief description will be given here, and the details will be referred to the above description.

  FIG. 14 is a flowchart of the RACP decoding program. In software processing by a computer, sequential processing as shown in FIG. 4 is fundamental. First, in step 21, the DC data DC0 compressed data of the image is subjected to DC decoding, that is, the reverse processing of the encoding processing performed at the time of data compression, thereby the average pixel of the 8 × 8 block. The value DC0 is restored and stored in the buffer.

  In steps 22 and 23, similarly to the RACP encoding program, the layer number LN is set to 1 (initial value) and the block number BN is set to 1 (initial value).

  In step 24, with respect to the block S designated by the block number BN, the center pixel values T and L of the upper block and the left block in the processed peripheral region are calculated by the linear prediction based on the above-described Expression 1.

  In step 25, the AC component prediction is performed on the block S designated by the block number BN based on the above-described equation 2, whereby the predicted pixel values C (= c00) of the sub-blocks s00 to s11 obtained by subdividing the block S are obtained. ˜c11) is calculated. In the subsequent step 26, the compressed data of the AC component ACn of the image is expanded, thereby restoring the prediction residual PR '. In step 27, the average pixel value DCn of the sub-block is calculated by adding the prediction residual PR ′ calculated in step 27 to the prediction pixel value C calculated in step 25, and is stored in the buffer. The

  In step 28, it is determined whether or not the block number BN has reached the end block number BNend. Until the end block number BNend is reached, the block number BN is incremented by 1 in step 29, and the processing in steps 24-27 is repeated. When the end block number BNend is reached, that is, when the processing of all the blocks in the image plane is completed, the process exits the loop from the determination result of step 28 and proceeds to step 30.

  In step 30, it is determined whether or not the hierarchy number LN has reached 3 (the lowest hierarchy). Until it reaches 3, the layer number LN is incremented by 1 in step 31, and the loop of steps 23 to 29 is repeated. As a result, the size of the block S to be processed gradually decreases as the hierarchy becomes lower, and the series of processes described above are recursively executed. The average pixel value DCn (output) of the sub-block calculated on the upper layer side is used as the average pixel value DCn-1 (input) of the block to be processed on the lower layer side and the unprocessed peripheral region. When the block number BN reaches 3, that is, when the processing of the lowest layer is finished, the loop is exited, and the processing of this program is finished.

  According to the RACP decoding program according to the present embodiment, the same effects as those of the above-described RACP decoder can be obtained.

  As is clear from the above description, according to the present embodiment, the prediction pixels of the sub-blocks s00 to s11 obtained by subdividing the block S using the spatial correlation of the peripheral regions located around the processing target. A value C (= c00 to c11) is calculated. At that time, among the peripheral areas to be referred to, for the processed peripheral areas that have already been processed by the sequential shift, the center pixel values CT and CL are used instead of the average pixel values DCn−1 of the blocks T and L. It is done. The center pixel values CT and CL are the average pixel values DCn of the sub-blocks t01, t11, l10, and l11 in the processed peripheral region and the average of the block S in consideration of the distance between the centers of blocks and sub-blocks having different sizes. It is specified as a linear prediction value based on the pixel value DCn-1. Therefore, compared with the conventional method in which the average of the average pixel values of the small blocks is assumed to be the center pixel value of the large block, the center pixel values CT and CL closer to the true value can be calculated. Further improvement can be achieved.

  In the above-described embodiment, AC component prediction encoding, particularly recursive AC component prediction encoding, has been described. However, the present invention is not limited to this, and an image with high resolution using AC component prediction is described. It is also applicable to image processing for generating

Overall configuration of RACP encoder Illustration of blocks set on the image plane Operation timing chart for parallel processing Operation timing chart in sequential processing Hierarchy processing block diagram Illustration of block center distance Calculation diagram of center pixel value by linear prediction Explanatory drawing of calculation of center pixel value by conventional method Explanatory drawing of alternating current component prediction encoding in this embodiment RACP encoding program flowchart Overall configuration of RACP decoder Hierarchy processing block diagram Illustration of AC component predictive decoding RACP decoding program flowchart

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 DC calculation part 2 DC encoding part 3 (3a-3c) Hierarchical processing part 5 DC decoding part 6 (6a-6c) Hierarchical processing part 31, 61 Area | region specific | specification part 32, 62 Prediction processing part 32a, 62a Center value prediction Unit 32b, 62b AC component prediction unit 33 subtractor 34 lossy conversion unit 35 entropy encoding unit 36, 64 inverse conversion unit 37, 63 adder

Claims (12)

In an image processing system that sets an plurality of first blocks on an image plane and performs AC component prediction while sequentially shifting the first block to be processed in a predetermined direction.
Among the peripheral areas located around the processing target, an area that has not been processed by sequential shift is specified as an unprocessed peripheral area, and an area that has already been processed by sequential shift is defined as a processed peripheral area An area identification part to be identified;
AC using the average pixel value of the first block to be processed, the central pixel value of the first block in the processed peripheral area, and the average pixel value of the first block in the unprocessed peripheral area A prediction processing unit that calculates a predicted pixel value of a second block obtained by subdividing the first block to be processed by component prediction;
The central pixel value uses an average pixel value of the second block in the processed peripheral area and an average pixel value of the first block to be processed, and the first block and the second block An image processing system characterized by being a linear prediction value based on the center-to-center distance.   The image processing system according to claim 1, wherein a second block positioned adjacent to the processing target is used as the second block in the processed peripheral area. An adder that calculates an average pixel value of the second block by adding a prediction residual corresponding to a difference from the true pixel value to the prediction pixel value calculated by the prediction processing unit;
2. The average pixel value of the second block calculated by the adder is fed back to the prediction processing unit as information on the processed peripheral area required for subsequent processing. Or the image processing system described in 2. As the hierarchy becomes lower, it has a plurality of hierarchy processing units in which the size of the first block to be processed decreases stepwise,
Each of the hierarchical processing units includes the region specifying unit, the prediction processing unit, and the adder,
The average pixel value of the second block calculated by the adder on the upper layer side is the average pixel value on the lower layer side as the average pixel value of the processing target on the lower layer side and the first block that is the unprocessed peripheral region. The image processing system according to claim 3, wherein the image processing system is supplied to the prediction processing unit. The image processing system is an encoder for compressing an image,
A subtractor that calculates a difference between a predicted pixel value calculated by the prediction processing unit and a true pixel value as a prediction residual;
An irreversible transformation unit that performs irreversible transformation on the prediction residual calculated by the subtractor;
An entropy encoding unit that generates an alternating current component of an image as a part of compressed data by performing entropy encoding on the prediction residual subjected to the irreversible transformation;
An inverse conversion unit that generates the prediction residual to be supplied to the adder by performing inverse processing of the lossy conversion on the prediction residual subjected to the lossy conversion. The image processing system according to claim 3 or 4, wherein the image processing system is characterized in that: The image processing system is a decoder that decompresses an image,
It further has an inverse transform unit that restores the prediction residual to be supplied to the adder by performing inverse processing of lossy transform and entropy coding performed at the time of image compression on the compressed data of the image. The image processing system according to claim 3 or 4, characterized by the above-mentioned. In an image processing program for setting an plurality of first blocks on an image plane and performing AC component prediction while sequentially shifting the first block to be processed in a predetermined direction,
A first step of specifying, as an unprocessed peripheral area, an area that has not yet been sequentially processed by a shift among peripheral areas located around the processing target;
A second step of identifying, as the processed peripheral area, an area in which processing by sequential shift has already been performed among the peripheral areas;
AC using the average pixel value of the first block to be processed, the central pixel value of the first block in the processed peripheral area, and the average pixel value of the first block in the unprocessed peripheral area Causing the computer to execute image processing including a third step of calculating predicted pixel values of a second block obtained by subdividing the first block to be processed by component prediction;
The central pixel value uses an average pixel value of the second block in the processed peripheral area and an average pixel value of the first block to be processed, and the first block and the second block An image processing program characterized by being a linear prediction value based on the center-to-center distance.   The image processing system according to claim 7, wherein a second block positioned adjacent to the processing target is used as the second block in the processed peripheral area. A fourth step of calculating an average pixel value of the second block by adding a prediction residual corresponding to a difference from the true pixel value to the calculated prediction pixel value;
9. The image processing program according to claim 7, wherein the calculated average pixel value of the second block is used as information on a processed peripheral area in subsequent processing. In the hierarchical structure in which the size of the block to be processed becomes smaller in stages as the hierarchy becomes lower, the image processing is recursively executed,
The average pixel value of the second block calculated on the upper layer side is used as an average pixel value of the first block serving as the processing target and the unprocessed peripheral region on the lower layer side. 9. An image processing program according to 9. The image processing program is an encoding program for compressing an image,
A fifth step of calculating a difference between the calculated predicted pixel value of the second block and the true pixel value as a prediction residual;
A sixth step of performing an irreversible transformation on the calculated prediction residual;
A seventh step of generating an alternating current component of an image as a part of compressed data by performing entropy coding on the prediction residual subjected to the irreversible transformation;
The prediction residual used for calculating the average pixel value of the second block is generated by performing inverse processing of the lossy conversion on the prediction residual subjected to the lossy conversion. The image processing program according to claim 9, further comprising a step. The image processing program is a decoding program for expanding an image,
Reconstructing the prediction residual used for calculating the average pixel value of the second block by performing inverse processing of irreversible transformation and entropy coding performed at the time of image compression on the compressed data of the image The image processing program according to claim 9 or 10, further comprising 9 steps.

Images