JP2011071825A - System and program for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance the spatial correlation of a reference region, and to further improve the prediction accuracy of an image in an AC component prediction. <P>SOLUTION: A region specifying part 31 specifies a plurality of inner blocks adjacent to an object block and outer blocks adjacent to the outside of the inner blocks in a plurality of predictive directions tying each of the inner blocks and the object block as the reference regions for the object block as object to be processed in processing at this time. A prediction processing part 32 predicts proximity pixel values at proximity predictive points nearer to the object block than the central positions of the inner blocks regarding each of the plurality of predictive directions by using at least the mean pixel value of the inner blocks and the mean pixel value of the outer blocks and on the basis of distances among the central positions of the blocks and the distances among the central positions of the blocks and the proximity predictive points. AC components are predicted by using the proximity pixel values predicted at each predictive direction. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing system and an image processing program, and more particularly, to AC component prediction (ACP).

  Conventionally, an image processing method called AC component prediction (ACP) is known. AC component prediction refers to a method for obtaining information on a small area obtained by subdividing a pixel block by referring to information on the peripheral area of the pixel block to be processed, and using spatial correlation with the surrounding area. It is a thing. For example, Patent Documents 1 to 3 disclose recursive alternating current component predictive coding that sequentially subdivides blocks to be processed and hierarchically encodes image data. Specifically, first, the predicted pixel values of sub-blocks obtained by subdividing the block to be processed are sequentially calculated on the image plane while sequentially shifting the block to be processed in a predetermined direction. This predicted pixel value is calculated by AC component prediction with reference to the peripheral region to be processed. Next, a difference between the predicted pixel value and the original pixel value (true value) is calculated as a prediction residual. Then, an irreversible transformation and entropy coding are performed on the prediction residual, thereby generating an AC component of an image that is a part of the compressed data. The above processing is performed by, for example, an 8 × 8 pixel block (top layer), a 4 × 4 pixel block (second layer), a 2 × 2 pixel block (third layer), and 1 × 1. It is recursively repeated in a hierarchical structure consisting of pixel blocks (lowest hierarchy).

  In particular, in Patent Document 1, in order to improve the prediction accuracy, among the reference regions that are referred to in the AC component prediction, for regions that have already been processed by the above-described sequential shift, sub-blocks with higher correlation are provided. A technique for referring to information is disclosed. For example, when a block of 4 × 4 pixels is processed in a line-sequential scan in the horizontal direction, blocks adjacent to the upper and left sides of the block to be processed are processed. In this case, the processed block is not the average pixel value of the block itself, but of the 2 × 2 pixel sub-blocks obtained by subdividing the block into four average pixels of the two sub-blocks adjacent to the processing target Refers to the value. The average of the two average pixel values is assumed to be the “center pixel value” of the 8 × 8 pixel block, and the AC component prediction is performed on the assumption of this. Here, the “center pixel value” refers to the true pixel value at the center position of the pixel block at infinite resolution.

Japanese Patent No. 4000157 Japanese Patent No. 3774201 Japanese Patent No. 3700906

  It is preferable that the image prediction accuracy by the AC component prediction is high. This is because the higher the prediction accuracy, the higher the correlation between the predicted image as a set of predicted pixel values and the original image, and the smaller the prediction residual corresponding to the difference between the two. The reduction of the prediction residual means that the tendency of the appearance frequency of the prediction residual statistically viewed tends to be close to 0. Therefore, for example, when entropy coding is performed on the prediction residual to compress the data, the compression rate can be increased by reducing the prediction residual. In AC component prediction, the closer the distance from the target block is, that is, the higher the spatial correlation, the smaller the divisor of the change amount (slope) of the pixel value, and the more accurate prediction becomes possible. In this regard, in Patent Documents 1 to 3, since the average pixel value of a block (Patent Document 1 is a sub-block) is used as the central pixel value of the block, and this value is used as it is as an input for AC component prediction, the prediction accuracy is improved. There was room to plan.

  Therefore, an object of the present invention is to increase the spatial correlation of the reference region and further improve the image prediction accuracy in the AC component prediction.

  In order to solve this problem, the first invention is to perform image processing in which a plurality of blocks having a predetermined size are set on an image plane, and blocks to be processed are sequentially shifted in a predetermined direction, and an alternating current component of the block is predicted. Provide a system. This image processing system includes an area specifying unit and a prediction processing unit. The area specifying unit, as a reference area of the target block to be processed in the current process, has a plurality of inner blocks adjacent to the target block and the outer side of the inner block in a plurality of prediction directions connecting each of the inner blocks and the target block. And the outer block adjacent to. The prediction processing unit, for each of a plurality of prediction directions, the neighborhood pixel value of the neighborhood prediction point closer to the target block than the center position of the inner block, at least the average pixel value of the inner block and the average pixel value of the outer block And predicting based on the distance between the center positions of the blocks and the distance between the center position of the blocks and the neighboring prediction points. In addition, the prediction processing unit calculates predicted pixel values of a plurality of sub-blocks obtained by subdividing the target block by AC component prediction using neighboring pixel values predicted for each prediction direction.

  Here, in the first invention, the neighboring pixel value is multiplied by the change rate of the pixel value per unit distance between the inner block and the outer block by the distance between the center position of the inner block and the neighboring prediction point. The value obtained by this multiplication may be calculated by adding to the average pixel value of the inner block. Alternatively, the neighboring pixel value is multiplied by the change rate of the pixel value per unit distance between the target block and the outer block by the distance between the center position of the inner block and the neighboring prediction point. You may calculate by adding to the average pixel value of an inner block what was obtained by multiplication. Furthermore, it is preferable that the distance between the center position of the inner block and the neighboring prediction point is the same for a pair of prediction directions arranged in a one-dimensional direction around the target block.

  In the first invention, an adder for calculating an average pixel value of the sub-block by adding a prediction residual corresponding to a difference from the true pixel value to the prediction pixel value predicted by the prediction processing unit. It may be provided.

  The first invention can be applied to a hierarchical structure in which the block size is reduced stepwise as the hierarchy becomes lower. In this case, the average pixel value of the sub-block calculated in the upper hierarchy is supplied to the lower hierarchy as the average pixel value of the block in the lower hierarchy located immediately below the upper hierarchy.

  The first invention can be applied to an encoder that compresses an image. The encoder includes a subtractor that calculates a difference between a predicted pixel value calculated by the prediction processing unit and a true pixel value as a prediction residual, and an irreversible conversion for the prediction residual calculated by the subtractor. An entropy encoding unit that generates an AC component of an image as a part of compressed data by performing entropy encoding on the prediction residual subjected to the irreversible conversion, It further includes an inverse transform unit that generates a prediction residual to be supplied to the adder by performing inverse processing of the irreversible transformation on the prediction residual subjected to the irreversible transformation.

  The first invention can also be applied to a decoder that expands an image. The decoder further includes an inverse transform unit that restores a prediction residual to be supplied to the adder by performing inverse processing of lossy transform and entropy coding performed at the time of image compression on the compressed data of the image. Have.

  The second invention provides an image processing program that sets a plurality of blocks of a predetermined size on an image plane and predicts an alternating current component of the blocks while sequentially shifting the blocks to be processed in a predetermined direction. This image processing program uses a plurality of inner blocks adjacent to the target block as a reference area of the target block to be processed in the current process, and the inner block in a plurality of prediction directions connecting each of the inner blocks and the target block. The step of identifying the outer block adjacent to the outer side, and for each of a plurality of prediction directions, the neighboring pixel value of the neighboring prediction point closer to the target block than the center position of the inner block, the average pixel value of the inner block, and the outer side Predicting at least using the average pixel value of the block, and predicting based on the distance between the central positions of the blocks and the distance between the central position of the block and the neighboring prediction points, Predicted pixels of multiple sub-blocks that subdivide the target block by AC component prediction using neighboring pixel values To execute the image processing method and a step of calculating the computer.

  Here, in the second invention, the neighboring pixel value is multiplied by the change rate of the pixel value per unit distance of the inner block and the outer block by the distance between the center position of the inner block and the neighboring prediction point, You may calculate by adding the value obtained by multiplication to the average pixel value of an inner block. Alternatively, the neighboring pixel value is multiplied by the rate of change of the pixel value per unit distance of the target block and the outer block by the distance between the center position of the inner block and the neighboring prediction point, and this multiplication is performed. The obtained value may be calculated by adding to the average pixel value of the inner block. Furthermore, it is preferable that the distance between the center position of the inner block and the neighboring prediction point is the same for a pair of prediction directions arranged in a one-dimensional direction around the target block.

  In the second invention, a step of calculating an average pixel value of the sub-block by adding a prediction residual corresponding to a difference from the true pixel value to the prediction pixel value may be further provided.

  The second invention can be applied to a hierarchical structure in which the block size gradually decreases as the hierarchy becomes lower. In this case, the image processing method is recursively executed, and the average pixel value of the sub-block calculated in the upper layer is supplied to the lower layer as the average pixel value of the block in the lower layer located immediately below the upper layer. The

  The second invention can be applied to an encoding program for compressing an image. The encoding program includes a step of calculating a difference between a predicted pixel value of a sub-block and a true pixel value as a prediction residual, performing an irreversible transformation on the calculated prediction residual, and irreversible By performing entropy coding on the transformed prediction residual, the step of generating the AC component of the image as a part of the compressed data and the prediction residual subjected to the irreversible transformation And causing the computer to execute an image processing method further including a step of generating a prediction residual to be supplied to the adder by performing an irreversible conversion inverse process.

  The second invention can also be applied to a decoding program for expanding an image. This decoding program restores the prediction residual used to calculate the average pixel value of the sub-block by performing reverse processing of lossy transformation and entropy encoding performed during image compression on the compressed data of the image And causing the computer to execute an image processing method.

  According to the first and second inventions, the information on the neighborhood prediction points is predicted from the information on the inner and outer blocks. Information on sub-blocks is not necessarily required, only information on blocks larger than this, and information with a resolution higher than ½ of one side of the block, that is, a neighboring prediction point closer to the target block than the center position of the inner block. Information can be obtained. The neighborhood prediction point is closer to the target block than the center position of the inner block, and tends to have a higher spatial correlation with the target block. Therefore, if the information on the neighboring prediction points obtained by the prediction is used for the AC component prediction, it is possible to further improve the image prediction accuracy.

Overall configuration of RACP encoder Illustration of blocks set on the image plane Operation timing chart for parallel processing Operation timing chart in sequential processing Configuration diagram of hierarchical processing unit in RACP encoder Explanatory diagram of positional relationship between block center position and nearby prediction points Explanation of calculation of neighboring pixel values using three points (λ = 1/2) Explanation of calculation of neighboring pixel values using three points (λ = 1/4) Explanation of calculation of neighboring pixel values using three points (λ = 1/8) Explanation of calculation of neighboring pixel values using two points (λ = 1/2) RACP encoding program flowchart Overall configuration of RACP decoder Configuration diagram of hierarchical processing unit in RACP decoder RACP decoding program flowchart

  Hereinafter, as an embodiment to which the AC component prediction according to the present invention is applied, recursive AC component prediction coding (RACP: Recursive ACP) will be described as an example.

(RACP encoder)
FIG. 1 is an overall configuration diagram of a RACP encoder. This encoder has a DC calculation unit 1, a DC encoding unit 2, and hierarchical processing units 3a to 3c, and is configured by four layers in this embodiment. Data TDCn, DCn (n = 0, 1, 2) output from these units 1, 2, 3a to 3c are temporarily stored in a buffer (storage unit) not shown.

  The DC calculation unit 1 divides an input image to be compressed into blocks having a preset size, and outputs the average pixel value of each block, that is, the average value of pixels included in the block, as TDC0 to TDC3. . Here, TDC0 is an average pixel value of a block of 8 × 8 pixels (hereinafter referred to as “8 × 8 block”), TDC1 is an average pixel value of a block of 4 × 4 pixels (hereinafter referred to as “4 × 4 block”), and TDC2 Is an average pixel value of a 2 × 2 pixel block (hereinafter referred to as “2 × 2 block”). TDC3 is an average pixel value of a 1 × 1 pixel block (hereinafter referred to as “1 × 1 block”), that is, a pixel value itself of a pixel which is a minimum unit of an image. The average pixel values TDC0 to TDC3 (true values) directly calculated from the input image are distinguished from the average pixel values DC0 to DC2 (restored values) restored through the processing in the units 2 and 3a to 3c. Please note that Since both values are accompanied by irreversible transformation in the process of encoding, they do not exactly match.

  The DC encoding unit 2 and the hierarchical processing units 3a to 3c include 8 × 8 blocks (top layer), 4 × 4 blocks (second layer), 2 × 2 blocks (third layer), and 1 × 1 block. In the four-layer structure composed of (the lowest layer), the layer processing assigned to itself is performed.

  The DC encoding unit 2 in the highest layer performs differential pulse code modulation (DPCM) and entropy encoding on the average pixel value TDC0 for each 8 × 8 block read from the buffer. Difference pulse code modulation encodes the difference between the average pixel values TDC0 for blocks adjacent to each other on the image plane. Then, entropy coding such as Huffman coding or arithmetic coding is performed on the coded difference after quantization. The data that has undergone such processing is output as a DC component DC0 of an image that is a part of the compressed data, and an average pixel value of an 8 × 8 block so as to be supplied to the hierarchical processing unit 3a of the immediately lower hierarchy. Is output as a value DC0 restored (inverse quantized).

  The three hierarchical processing units 3a to 3c include an average pixel value TDCn (n = 1, 2, 3, and so on) generated by the DC calculation unit 1 and an average pixel value DCn−1 generated by the immediately higher hierarchy. As a part of the compressed data, the AC component ACn of the image is output by the process including the AC component prediction. At the same time, the hierarchical processing units 3a to 3b (except 3c) restore the average pixel value DCn from the AC component ACn. The restored average pixel values DC0, 1, 2 are output for supply to the immediately lower hierarchy.

  Specifically, the hierarchical processing unit 3a in the second hierarchical level undergoes processing such as AC component prediction and prediction residual calculation for 8 × 8 blocks, and 4 × 4 blocks of AC components AC1 and 4 The average pixel value DC1 of x4 blocks is output. In this hierarchy, the average pixel value DC0 of 8 × 8 blocks generated in the highest hierarchy is used as information to be referenced in the AC component prediction. At the same time, an average pixel value TDC1 of 4 × 4 blocks calculated by the DC calculation unit 1 is input to calculate a prediction residual. The hierarchical processing unit 3b of the third hierarchical level performs processing such as AC component prediction on 4 × 4 blocks as processing targets, and outputs 2 × 2 blocks of AC components AC2 and 2 × 2 blocks of average pixel values DC2. Output. In this hierarchy, the average pixel value DC1 of 4 × 4 blocks generated in the second hierarchy is used as reference information for AC component prediction. At the same time, an average pixel value TDC2 of 2 × 2 blocks calculated by the DC calculation unit 1 is input to calculate a prediction residual. The hierarchy processing unit 3c in the lowest hierarchy outputs an AC component AC3 of 1 × 1 block through processing such as AC component prediction for 2 × 2 blocks (calculation of DC3 is not necessary). In this hierarchy, the average pixel value DC2 of 2 × 2 blocks generated in the third hierarchy is used as reference information for AC component prediction. At the same time, the average pixel value TDC3 of 1 × 1 block calculated by the DC calculation unit 1 is used to calculate the prediction residual.

  As described above, in the hierarchical structure in which the sizes of the blocks to be processed are different, the DC encoding unit 2 and the hierarchical processing units 3a to 3c are linked to each other so that image processing mainly based on AC component prediction is recursively executed. Is done. As a result, the DC component DC0 of the image and the AC components AC1 to AC3 are output as compressed data. The compressed data includes accompanying information such as a Huffman table in addition to these.

  FIG. 2 is an explanatory diagram of blocks set on the image plane. A plurality of blocks are set on the image plane by dividing an image to be compressed (one frame image or a partial image thereof) vertically and horizontally, and processed in units of blocks. The block size is set so as to decrease step by step as the hierarchy becomes lower. The size of the block to be processed in a certain hierarchy matches the size of the sub-block subdivided in the hierarchy located immediately above it. In other words, the size of the upper-layer sub-block matches the size of the lower-layer block located immediately below it. Processing of the entire image in a certain hierarchy is achieved by repeating the processing while sequentially shifting the blocks to be processed on the screen and processing all the blocks in the image. As shown in the figure, the shift direction (scan direction) may be a line-sequential scan along the horizontal direction, but may be arbitrarily set including the one along the vertical direction. Further, the shift directions in the respective hierarchies are not necessarily the same. In the case of the illustrated shift direction, if the block to be processed is “S”, the upper side of the horizontal line to which the target block S belongs is the processed area, and the lower side is the unprocessed area. As for the horizontal line of the target block S, the left side of the target block S is a processed area, and the right side is an unprocessed area. As reference areas to be referred to in the processing of the target block S, eight blocks T, TT, L, LL, B, BB, R, and RR (hereinafter referred to as “T to RR”) having the same block size are used. . Among these reference blocks, the blocks T, L, B, and R are particularly referred to as “inner blocks”, and the blocks TT, LL, BB, and RR are referred to as “outer blocks”. The inner block T is adjacent to the upper side of the target block S, and the outer block TT is adjacent to the outer side of the inner block T in the upper prediction direction connecting the inner block T and the target block S. In a similar relationship, the blocks L and LL are adjacent to each other in the left prediction direction, the blocks B and BB are adjacent to each other in the lower prediction direction, and the blocks R and RR are adjacent to each other in the right prediction direction. Based on the AC component prediction of the target block S, the average pixel value of the sub-block divided into four is calculated. Hereinafter, when subblocks are subdivided into blocks, the upper left subblock is identified by the subscript “00”, the upper right is “01”, the lower right is “11”, and the lower left is “10”. As a reference area of the target block S, not only blocks T, L, B, and R that are directly adjacent to this, but also blocks TT, LL, BB, and RR separated by one block are used together, and reference blocks having the same block size. The AC component prediction is performed with reference to only the information on T to RR (however, subblock information may be used in a complementary manner).

  In the present embodiment, eight reference blocks T to RR are set, but this is only an example, and other setting methods may be used. For example, only a part of these, or other additions (diagonal direction, one jump, etc.). In this case, as in the case of FIG. 2, it is preferable that the reference area of the target block S is opposed to the target block S on the image plane.

  The processing of the RACP encoder may be either parallel processing or sequential processing. FIG. 3 is an operation timing chart in the parallel processing. First, the DC calculation unit 1 operates to generate average pixel values TDC0 to TDC3. Then, as the generation of all the average pixel values TDC0 to TDC3 is completed, the DC encoding unit 2 and the hierarchical processing units 3a to 3c operate in parallel. However, the timing at which these operations start is not the same, and the lower the hierarchy, the later the start timing. This delay is caused by processing delay due to sequential shift of the upper layer. Referring to FIG. 2, when processing a block S in a certain hierarchy, the condition (hierarchical process start condition) is that the processing of the reference blocks T to RR is finished in the immediately higher hierarchy. In other words, the processing of the block S in the lower layer cannot be started unless the sequential shift proceeds and the processing up to the block BB is finished in the upper layer. This is the reason why the operation delay occurs in the lower layer. However, compared with the sequential processing as shown in FIG. 4, the pipelined parallel processing shown in FIG. 3 is faster. On the other hand, in the sequential processing of FIG. 4, the processing of the immediately lower hierarchy is started when all of the processing of a certain hierarchy is completed. Accordingly, the hierarchical processing start condition is naturally satisfied even in the sequential processing.

  FIG. 5 is a configuration diagram of the hierarchy processing unit 3 (generic name for 3a to 3c) in the RACP encoder. The hierarchy processing unit 3 in each hierarchy is basically the same in configuration and processing flow except that the block size to be handled is different (however, for the lowest hierarchy processing unit 3c, the generation system 36 of DC3). 37 is unnecessary). The hierarchical processing unit 3 includes a region specifying unit 31, a prediction processing unit 32, a subtractor 33, an irreversible conversion unit 34, an entropy encoding unit 35, an inverse conversion unit 36, and an adder 37.

  The area specifying unit 31 specifies the target block S to be processed this time and the eight reference blocks T to RR. The information of the reference blocks T to RR is not related to whether the block has been processed or not processed, in other words, regardless of whether or not the information of the sub-block in the block is calculated. The average pixel value DCn−1 calculated in step 1 is uniformly used. The average pixel value DCn−1 read from the buffer is supplied to the prediction processing unit 32.

  The prediction processing unit 32 performs AC component prediction of the target block S using the average pixel value DCn-1 of the target block S and the average pixel value DCn-1 of each of the blocks T to RR. At that time, for each of the upper, lower, left, and right prediction directions, a neighborhood prediction point is set at a position closer to the target block S than the center position of the inner block, and a neighborhood pixel value at this neighborhood prediction point is predicted and calculated. The AC component prediction is performed using the neighboring pixel values calculated individually for each prediction direction. Thereby, the predicted pixel values of the four sub-blocks s00 to s11 obtained by subdividing the target block S are calculated.

  The prediction processing unit 32 calculates predicted pixel values C (= c00 to c11) of sub-blocks s00 to s11 obtained by subdividing the target block S, that is, predicted images, based on the following formulas 1 and 2. Here, Formula 2 is a calculation formula of the neighboring pixel values L ′, R ′, T ′, and L ′ in the case of λ = 1/2, and α and β are the horizontal tilt and the vertical tilt, respectively. Show. Regarding λ, in the direction from the target block to the reference block, the block boundary between the target block and the inner block is λ = 0, and the distance from the block boundary to the center position of the inner block is λ = 1. The neighboring pixel values L ′, R ′, T ′, and B ′ are values predicted by linear prediction or the like for the neighboring prediction points that are closer to the target block than the center position of the inner block. In Equations 1 and 2 and the following description, the symbols “S”, “T”, “TT”, “L”, “LL”, “B”, “BB”, “R”, and “RR” are the blocks themselves. Rather than the average pixel value DCn-1 (center pixel value) of these blocks. The calculated predicted pixel value C = {c00, c01, c10, c11} is supplied to the subtracter 33 and the adder 37.

(Formula 1)
c00 = T + (α + β)
c01 = T + (− α + β)
c10 = T + (α−β)
c11 = T + (− α−β)

(Formula 2)
α = (L′−R ′) / 6
β = (T′−B ′) / 6
L ′ = L + (S−LL) / 8
R ′ = R + (S−RR) / 8
T ′ = T + (S−TT) / 8
B ′ = B + (S−BB) / 8

  FIG. 6 is an explanatory diagram of the positional relationship between the block center position and the vicinity prediction point at λ = 1/2. The center pixel value at the center position of each block is regarded as the average pixel value DCn-1 of the block itself. In order to match the scale of λ, the length of one side of the square when the block is divided into four is set to 1, and the length of one side of the square when the sub-block is further divided into four is set to ½. The center-to-center distance between adjacent blocks is 2, and the distance between the center position of the target block and each neighboring prediction point (its neighboring pixel values = L ′, R ′, T ′, B ′) is 3/2. Become. For each prediction direction, the neighborhood pixel value (eg, R ′) of the neighborhood prediction point uses at least the center pixel value (eg, R) of the inner block and the center pixel value (eg, RR) of the outer block, and Prediction / calculation is performed based on the distance between the center positions of each other and the distance between the center position of the block and the neighboring prediction points.

  FIG. 7 is an explanatory diagram for calculating the neighboring pixel value R ′ in the right prediction direction using three points at λ = 1/2. First, the inclination θ is calculated by dividing the difference between the center pixel value S of the target block and the center pixel value RR of the right outer block by the center-to-center distance 4 of these blocks. This inclination θ corresponds to the rate of change of the pixel value per unit distance between the target block and the right outer block. Next, on the premise that the pixel values change uniformly in the same prediction direction, the distance θ between the center position of the right inner block and the right neighboring prediction point is set to the gradient θ in the right prediction direction. Is multiplied by a change amount Δp from the center pixel value R of the right inner block. Then, by adding the change amount Δp to the central pixel value R of the right inner block, the neighboring pixel value R ′ of the right neighboring prediction point is calculated. In the same manner, the neighborhood pixel values T ′, L ′, and B ′ of the neighborhood prediction points that are shared by λ = ½ at the upper side are calculated.

  From the spatial correlation of the image, the neighboring pixel values T ′ to R ′ of the neighboring prediction points close to the target block reflect the characteristics of the target block more deeply than the center pixel values T to R of the inner block farther than this. Can be considered closer to the true value. Therefore, by using the neighboring pixel values T ′ to R ′ instead of the conventional center pixel values T to R, it is possible to perform AC component prediction with higher prediction accuracy. When a part of the peripheral area is missing like an edge on the image plane, the prediction processing unit 32 exceptionally uses the center pixel value S of the target block as information on the part. In the actual processing, it is natural that the above formulas 1 and 2 may be integrated and handled as a single formula.

  In the present embodiment, neighboring pixel values T ′ to R ′ closer to the target block are predicted (typically linear prediction) with a resolution (λ <1) higher than ½ of one side of the block, and this is used. It is characterized in that AC component prediction is performed. Therefore, the value of λ may be set to an arbitrary value that satisfies λ <1 other than ½. In particular, if λ is set to a value smaller than 1/2 (such as 1/4 or 1/8), the accuracy higher than the resolution of the sub-block can be ensured. FIG. 8 is an explanatory diagram for calculating the neighboring pixel values T ′ to R ′ using three points at λ = 1/4. In this case, the neighboring pixel values T ′ to R ′ and the gradients α and β are determined from a linear predictive approach that takes into account the distance between the center positions of the blocks and the distance between the center position of the blocks and the neighboring prediction points. , Calculated from Equation 3.

(Formula 3)
α = (L′−R ′) / 5
β = (T′−B ′) / 5
L ′ = L + (S−LL) × 3/16
R ′ = R + (S−RR) × 3/16
T ′ = T + (S−TT) × 3/16
B ′ = B + (S−BB) × 3/16

  FIG. 9 is an explanatory diagram of calculation of neighboring pixel values using three points at λ = 1/8. In this case, the neighboring pixel values T ′ to R ′ and the inclinations α and β are calculated from Equation 4.

(Formula 4)
α = (L′−R ′) × 4/18
β = (T′−B ′) × 4/18
L ′ = L + (S−LL) × 7/32
R ′ = R + (S−RR) × 7/32
T ′ = T + (S−TT) × 7/32
B ′ = B + (S−BB) × 7/32

  Further, instead of the above-described linear prediction using the three points, the neighboring pixel values T ′ to R ′ may be calculated by using the linear prediction using two points as shown in FIG. Taking λ = 1/2 as an example, for example, the neighboring pixel value R ′ is set to the rate of change of the pixel value per unit distance (inclination θ = (R−RR) / 2) of the inner right and outer right blocks, It is obtained by multiplying 1/2 which is the distance between the block center position and the neighboring prediction points, and adding the value Δp obtained by this multiplication to the center pixel value R of the inner block.

(Formula 5)
α = (L′−R ′) / 6
β = (T′−B ′) / 6
L ′ = (5L−LL) / 4
R ′ = (5R−RR) / 4
T ′ = (5T−TT) / 4
B ′ = (5B−BB) / 4

  Note that it is preferable to set the value of λ to be the same for a pair of prediction directions arranged in a one-dimensional direction around the target block, specifically, the upper and lower prediction directions or the left and right prediction directions. As a result, the distance between the center position of the inner block and the neighboring prediction point is also the same in these prediction directions. The reason why this is preferable is that the prediction accuracy can be improved due to the symmetry of the neighboring prediction points (that is, the positions of the upper and lower (or left and right) neighboring prediction points with respect to the target block are symmetric). Because. As long as this symmetry is maintained, the value of λ can be made different depending on the prediction direction, such as setting the value of λ on the left and right to ½ and the value of λ on the top and bottom to ¼. is there.

  From the viewpoint of prediction accuracy, three-point prediction (Formula 2) is superior to two-point prediction (Formula 5). However, the two-point prediction has an advantage that the amount of calculation can be reduced by a smaller number of samples than the three-point prediction. Further, as described above, the neighboring pixel value R ′ is calculated by adding the change amount Δp calculated from the inclination θ to the central pixel value R of the inner block. This inclination θ can be calculated by a difference method, and as a specific calculation method thereof, a forward difference (Δu (n) = u (n + 1) −u (n)) with emphasis on simplicity of calculation, There is a central difference (Δu (n) = (u (n + 1) −u (n−1)) / 2) that places importance on the accuracy of the inclination. These calculation methods may be a first-order difference, but may be a second-order difference or more in order to calculate a higher-precision gradient.

  Referring to FIG. 5 again, the subtractor 33 calculates the predicted pixel value C (for example, 4 × 4 block unit) calculated by the prediction processing unit 32 and the corresponding average pixel value TDCn (for example, 4 × 4 block unit). That is, the difference from the true pixel value calculated based on the input image is calculated as the prediction residual PR. The closer the predicted image (set of predicted pixel values c00 to c11) generated by AC component prediction is to the original image (true pixel value), the smaller the prediction residual PR. The smaller the prediction residual PR, the more likely the appearance frequency of the prediction residual statistically tends to be close to 0, which is advantageous in performing entropy coding.

  The irreversible transform unit 34 performs irreversible transforms such as Hadamard transform and quantization on the prediction residual PR calculated by the subtractor 33. The entropy coding unit 35 performs entropy coding such as Huffman coding or arithmetic coding on the prediction residual PR subjected to the irreversible transformation, thereby, for example, alternating current component ACn (4 × 4 block unit ACn ( Generate compressed data)). The inverse transform unit 36 calculates a prediction residual PR ′ by performing reverse processing of the lossy transformation on the prediction residual PR subjected to the lossy transformation. This prediction residual PR ′ is a restoration of the original prediction residual PR, but due to the irreversible transformation (the original value cannot be completely restored), the original prediction residual PR. Is slightly different. The adder 37 calculates the average pixel value DCn of the sub-block (for example, 4 × 4 block) by adding the prediction residual PR ′ to the prediction pixel value C calculated by the prediction processing unit 32. The reason why the restored prediction residual PR ′ is used as the input of the adder 37 instead of the original prediction residual PR is that accumulation of errors is prevented by iterative processing during decoding, and the reproducibility of the expanded image is ensured. It is to do. The average pixel value DCn output from the adder 37 is temporarily stored in the buffer, and is read out as needed in the processing in the immediately lower hierarchy.

  As described above, according to the RACP encoder according to the present embodiment, the information on the neighboring prediction points is predicted using at least the information on the inner and outer blocks. That is, in the 2-point prediction, information on the inner block and information on the outer block are used, and in the 3-point prediction, information on the target block is also used in addition to these. As a result, even if the sub-block information is not used, only the information of the block larger than this is used, and the resolution is higher than ½ of one side of the block, that is, the target block is more than the center position of the inner block. Information on near neighbor prediction points can be obtained. The neighborhood prediction point is closer to the target block than the center position of the inner block, and tends to have a higher spatial correlation with the target block. Therefore, if the AC component prediction is used for the information of the neighborhood prediction points having higher correlation, it is possible to further improve the image prediction accuracy. As a result, the prediction residual PR, which is a difference from the true pixel value, can be reduced, and further improvement in compression efficiency can be expected. In particular, if pipelined parallel processing as shown in FIG. 3 is performed, overall processing time can be reduced.

(RACP encoding program)
Next, a description will be given of a RACP encoding program for realizing processing equivalent to a RACP encoder realized as hardware by software processing of a computer. Since there is no essential difference between hardware processing and software processing, only a brief description is given here, and the details are given in the above description.

  FIG. 11 is a flowchart of the RACP encoding program. In software processing by a computer, sequential processing as shown in FIG. 4 is fundamental. First, in step 1, a plurality of blocks (4 types) are set on the image plane of the input image, and average pixel values TDC0 to TDC3 of each block are calculated. The calculated average pixel values TDC0 to TDC3 are stored in the buffer as needed. When all the blocks have been processed, the process proceeds to Step 2.

  In Step 2, DC coding, that is, differential pulse width modulation and entropy coding is performed on the average pixel value TDC0 of the 8 × 8 block, and thereby the DC component of the image that becomes part of the compressed data. DC0 is generated and output. The restored value is stored in the buffer as DC0.

  In step 3, the hierarchy number LN is set to 1 (initial value). LN = 1 indicates that the hierarchy to be processed is the highest hierarchy. Each time one hierarchical process is completed, it is incremented by 1 (step 14), and the process proceeds in order to the lower hierarchy. Then, when LN = 3, that is, when the processing of the lowest layer is finished, all the processing is finished (step 13).

  In step 4, the block number BN is set to 1 (initial value). With the setting of the layer number LN in the previous step 3, the size of the block to be processed in that layer is uniquely specified, and the end block number BNend corresponding to the total number of blocks is also specified. Each time processing of a block in the same hierarchy is completed, the block number BN is incremented by 1 (step 12), and the processing target is sequentially shifted in a predetermined direction. Then, when the processing of the block corresponding to the end block number BNend is finished, the processing in the hierarchy is finished (step 11).

  In step 5, with respect to the block S designated by the block number BN, the neighboring pixel values T ′ to R ′ of the neighboring prediction points in the four directions are calculated based on, for example, Equation 2.

  In step 6, the AC component prediction is performed on the block S designated by the block number BN based on Equation 1, and thereby, the predicted pixel values C (= c00˜) of the sub-blocks s00 to s11 obtained by subdividing the block S are obtained. c11) is calculated. In step 7 following this, the difference between the predicted pixel value C calculated in the previous step 6 and the true pixel value TDCn is calculated as the prediction residual PR.

  In step 8, irreversible transformation and entropy coding are performed on the prediction residual PR calculated in the previous step 7. As a result, an AC component ACn of the image as a part of the compressed data is generated. In subsequent step 9, the prediction error PR subjected to the irreversible transformation is subjected to the inverse process to calculate a prediction error PR ′ (restored value) obtained by restoring the original prediction error PR. In step 10, by adding the prediction residual PR ′ to the prediction pixel value C calculated in step 6, the average pixel value DCn of the sub-block is calculated and stored in the buffer. Note that the processing in steps 9 and 10 is skipped in LN = 3, that is, in the lowest hierarchical processing.

  In step 11, it is determined whether or not the block number BN has reached the end block number BNend. Until the end block number BNend is reached, the block number BN is incremented by 1 in step 12, and the processing in steps 5 to 11 is repeated. When the end block number BNend is reached, that is, when the processing of all the blocks in the image plane is completed, the process exits the loop and proceeds to step 13.

  In step 13, it is determined whether or not the hierarchy number LN has reached 3 (the lowest hierarchy). Until it reaches 3, the layer number LN is incremented by 1 in step 14, and the loop of steps 5 to 12 is repeated. As a result, the size of the block S to be processed gradually decreases as the hierarchy becomes lower, and the series of processes described above are recursively executed. The average pixel value DCn (output) of the sub-block calculated on the upper layer side is used as the average pixel value DCn-1 (input) on the lower layer side. When LN = 3 and BN = BNend is reached, that is, when all the processes in the lowest hierarchy are completed, the loop exits from the judgment results of steps 11 and 13, thereby The process ends.

  According to the RACP encoding program according to the present embodiment, as with the above-described RACP encoder, it is possible to further improve the prediction accuracy of an image in AC component prediction, and to expect an improvement in compression efficiency.

(RACP decoder)
FIG. 12 is an overall configuration diagram of a RACP decoder that decompresses compressed data generated by the above-described RACP encoder or RACP encoding program. This decoder is mainly composed of DC decoding 5 and three hierarchical processing units 6a to 6c. Data DCn (n = 0, 1, 2, 3) output from these units 5, 6a to 6c is temporarily stored in a buffer (storage unit) (not shown).

  The DC decoding 5 and the hierarchy processing units 6a to 6c are configured to have 8 × 8 blocks (the highest layer), 4 × 4 blocks (the second layer), 2 × 2 blocks (the third layer), and 1 × 1 block ( In the hierarchical structure of the lowest hierarchy, the hierarchical processing assigned to itself is performed. The DC decoding unit 5 in the highest hierarchy generates an average pixel value DC0 of 8 × 8 blocks by performing reverse processing of the processing applied at the time of image compression on the compressed data related to the DC component DC0 of the image. This is supplied to the hierarchy processing unit 6a of the second hierarchy.

  The hierarchical structure itself of the RACP decoder is substantially the same as that of the RACP encoder, but the AC components AC1 to AC3 that are the outputs of the hierarchical processing units 3a to 3c in the encoder are input to the hierarchical processing units 6a to 6c in the decoder. Is different. These hierarchical processing units 6a to 6c restore the average pixel value DCn based on the compressed data related to the AC components AC1 to AC3 of the image and the average pixel value DCn-1 supplied from the immediately higher hierarchy. The restored average pixel value DCn is output for supply to the immediately lower hierarchy as necessary, and is stored in the buffer. Then, a set of average pixel values DC3 of 1 × 1 blocks calculated by the lowest-level hierarchy processing unit 6c is the final decompressed image. Note that the direction of sequential shift in the decoder conforms to that of the encoder. The decoder processing may be either parallel processing or sequential processing.

  FIG. 13 is a configuration diagram of the hierarchical processing unit 6 (generic name of 6a to 6c) in the RACP decoder. The hierarchy processing unit 6 in each hierarchy has the same basic configuration and operation except that the size of blocks to be handled is different. The hierarchy processing unit 6 includes a region specifying unit 61, a prediction processing unit 62, an adder 63, and an inverse conversion unit 64.

  Similar to the RACP encoder, the region specifying unit 61 specifies the target block S to be processed this time and the reference blocks T to RR. That is, as shown in FIG. 2, the average pixel value DCn-1 of the blocks S and T to RR is read from the buffer. These average pixel values DCn−1 have already been calculated in the immediately higher hierarchy and are supplied from the immediately higher hierarchy via a buffer. The average pixel value DCn−1 read from the buffer is supplied to the prediction processing unit 62.

  The prediction processing unit 62 performs AC component prediction of the target block S using DCn−1 supplied from the buffer. As a result, predicted pixel values C (= c00 to c11) of the target sub-blocks s00 to s11 obtained by subdividing the target block S, that is, predicted images are calculated. The calculated predicted pixel value C is supplied to the adder 63. When a part of the peripheral area is missing like an edge on the image plane, it is processed according to a rule decided in advance with the decoder. For example, the average pixel value DCn-1 of the block S to be processed is supplemented as the information of that portion.

  The inverse transform unit 64 should supply the adder 63 by performing inverse processing of lossy transformation and entropy encoding performed at the time of image compression on the compressed data of the AC component ACn of 4 × 4 blocks, for example. The prediction residual PR ′ is restored. The adder 63 adds the prediction residual PR ′ reconstructed by the inverse transform unit 64 to the predicted pixel value C calculated by the prediction processing unit 62 to thereby obtain an average pixel of the sub-block (for example, 4 × 4 block). The value DCn is calculated and temporarily stored in the buffer. The average pixel value DCn stored in the buffer is also supplied to the prediction processing unit 62 in the immediately lower hierarchy as an average pixel value of a block (for example, 4 × 4 block) in the immediately lower hierarchy.

  As described above, according to the RACP decoder according to the present embodiment, the compressed data generated by the above-described RACP decoder or RACP encoding program can be appropriately decompressed. In particular, if pipelined parallel processing as shown in FIG. 3 is performed, overall processing time can be reduced.

(RACP decoding program)
Next, a RACP decoding program for realizing processing equivalent to the RACP decoder realized as hardware by software processing of a computer will be described. Since there is no essential difference between hardware processing and software processing, only a brief description will be given here, and the details will be referred to the above description.

  FIG. 14 is a flowchart of the RACP decoding program. In software processing by a computer, sequential processing as shown in FIG. 4 is fundamental. First, in step 21, the DC data DC0 compressed data of the image is subjected to DC decoding, that is, the reverse processing of the encoding processing performed at the time of data compression, thereby the average pixel of the 8 × 8 block. The value DC0 is restored and stored in the buffer.

  In steps 22 and 23, similarly to the RACP encoding program, the layer number LN is set to 1 (initial value) and the block number BN is set to 1 (initial value).

  In step 24, with respect to the block S designated by the block number BN, the neighboring pixel values T ′, L ′, B ′, and R ′ of the neighboring prediction points are calculated based on Equation 2, for example.

  In step 25, the AC component prediction is performed on the block S designated by the block number BN based on the above-described equation 1, whereby the predicted pixel values C (= c00) of the sub-blocks s00 to s11 obtained by subdividing the block S are obtained. ˜c11) is calculated. In the subsequent step 26, the compressed data of the AC component ACn of the image is expanded, thereby restoring the prediction residual PR '. In step 27, the average pixel value DCn of the sub-block is calculated by adding the prediction residual PR ′ calculated in step 27 to the prediction pixel value C calculated in step 25, and is stored in the buffer. The

  In step 28, it is determined whether or not the block number BN has reached the end block number BNend. Until the end block number BNend is reached, the block number BN is incremented by 1 in step 29, and the processing in steps 24-27 is repeated. When the end block number BNend is reached, that is, when the processing of all the blocks in the image plane is completed, the process exits the loop from the determination result of step 28 and proceeds to step 30.

  In step 30, it is determined whether or not the hierarchy number LN has reached 3 (the lowest hierarchy). Until it reaches 3, the layer number LN is incremented by 1 in step 31, and the loop of steps 23 to 29 is repeated. As a result, the size of the block S to be processed gradually decreases as the hierarchy becomes lower, and the series of processes described above are recursively executed. The average pixel value DCn (output) of the sub-block calculated on the upper layer side is used as the average pixel value DCn-1 (input) of the block to be processed on the lower layer side and the unprocessed peripheral region. When LN = 3 and BN = BNend is reached, that is, when the processing of the lowest layer is finished, the loop is exited, and the processing of this program is finished.

  According to the RACP decoding program according to the present embodiment, the same effects as those of the above-described RACP decoder can be obtained.

  In the above-described embodiment, AC component prediction encoding, particularly recursive AC component prediction encoding, has been described. However, the present invention is not limited to this, and an image with high resolution using AC component prediction is described. It is also applicable to image processing for generating

DESCRIPTION OF SYMBOLS 1 DC calculation part 2 DC encoding part 3 (3a-3c) Hierarchical processing part 5 DC decoding part 6 (6a-6c) Hierarchical processing part 31, 61 Area | region specific | specification part 32, 62 Prediction processing part 33 Subtractor 34 Nonreversible Conversion unit 35 Entropy encoding unit 36, 64 Inverse conversion unit 37, 63 Adder

Claims (16)

In an image processing system that sets a plurality of blocks of a predetermined size on an image plane and performs block alternating component prediction while sequentially shifting the blocks to be processed in a predetermined direction,
Adjacent to the outside of the inner block in a plurality of prediction directions connecting each of the inner block and the inner block as a reference area of the target block to be processed in the current processing, and the inner block. An area specifying part for specifying the outer block,
For each of the plurality of prediction directions, a neighboring pixel value of a neighboring prediction point that is closer to the target block than a center position of the inner block, an average pixel value of the inner block, and an average pixel value of the outer block are at least And predicting based on the distance between the center positions of the blocks and the distance between the center position of the blocks and the vicinity prediction point, and using the vicinity pixel value predicted for each prediction direction An image processing system comprising: a prediction processing unit that calculates predicted pixel values of a plurality of sub-blocks obtained by subdividing the target block by AC component prediction.   The neighboring pixel value is obtained by multiplying the rate of change of the pixel value per unit distance between the inner block and the outer block by the distance between the center position of the inner block and the neighboring prediction point. The image processing system according to claim 1, wherein the calculated value is calculated by adding the obtained value to an average pixel value of the inner block.   The neighboring pixel value is obtained by multiplying the rate of change of the pixel value per unit distance between the target block and the outer block by the distance between the center position of the inner block and the neighboring prediction point, and multiplying it. The image processing system according to claim 1, wherein the calculated value is calculated by adding the calculated value to an average pixel value of the inner block.   The distance between the center position of the inner block and the neighboring prediction point is the same with respect to a pair of prediction directions arranged in a one-dimensional direction around the target block. The image processing system described in 1.   An adder that calculates an average pixel value of the sub-block by adding a prediction residual corresponding to a difference from a true pixel value to the prediction pixel value predicted by the prediction processing unit; The image processing system according to claim 1, wherein the image processing system is characterized in that: In a hierarchical structure in which the size of the block decreases in stages as the hierarchy becomes lower,
6. The average pixel value of the sub-block calculated in an upper layer is supplied to the lower layer as an average pixel value of the block in a lower layer located immediately below the upper layer. The described image processing system. The image processing system is an encoder for compressing an image,
A subtractor that calculates a difference between the predicted pixel value calculated by the prediction processing unit and a true pixel value as a prediction residual;
An irreversible transformation unit that performs irreversible transformation on the prediction residual calculated by the subtractor;
An entropy encoding unit that generates an alternating current component of an image as a part of compressed data by performing entropy encoding on the prediction residual subjected to the irreversible transformation;
An inverse conversion unit that generates the prediction residual to be supplied to the adder by performing inverse processing of the lossy conversion on the prediction residual subjected to the lossy conversion. The image processing system according to claim 5 or 6, characterized by the above-mentioned. The image processing system is a decoder that decompresses an image,
It further has an inverse transform unit that restores the prediction residual to be supplied to the adder by performing inverse processing of lossy transform and entropy coding performed at the time of image compression on the compressed data of the image. The image processing system according to claim 5 or 6, characterized by the above-mentioned. In an image processing program that sets a plurality of blocks of a predetermined size on an image plane and performs block alternating current component prediction while sequentially shifting a block to be processed in a predetermined direction,
As a reference area of the target block to be processed in the current process, a plurality of inner blocks adjacent to the target block, and a plurality of prediction directions connecting each of the inner blocks and the target block, outside the inner block. Identifying adjacent outer blocks;
For each of the plurality of prediction directions, a neighboring pixel value of a neighboring prediction point that is closer to the target block than a center position of the inner block, an average pixel value of the inner block, and an average pixel value of the outer block are at least And predicting based on the distance between the center positions of the blocks and the distance between the center position of the blocks and the neighboring prediction points;
Calculating predicted pixel values of a plurality of sub-blocks obtained by subdividing the target block by AC component prediction using the neighboring pixel values predicted for each of the neighboring prediction points. An image processing program for causing a computer to execute the method.   The neighboring pixel value is obtained by multiplying the rate of change of the pixel value per unit distance between the inner block and the outer block by the distance between the center position of the inner block and the neighboring prediction point. The image processing program according to claim 9, wherein the image processing program is calculated by adding the obtained value to an average pixel value of the inner block.   The neighboring pixel value is obtained by multiplying the rate of change of the pixel value per unit distance between the target block and the outer block by the distance between the center position of the inner block and the neighboring prediction point, and multiplying it. The image processing program according to claim 9, wherein the calculated value is added to an average pixel value of the inner block.   The distance between the center position of the inner block and the neighboring prediction point is the same for a pair of prediction directions arranged in a one-dimensional direction around the target block. The image processing program described in 1.   The method according to claim 9, further comprising: calculating an average pixel value of the sub-block by adding a prediction residual corresponding to a difference from a true pixel value to the prediction pixel value. Any of the image processing programs described. The image processing method is recursively executed in a hierarchical structure in which the size of the block is gradually reduced as the hierarchy becomes lower,
The average pixel value of the sub-block calculated in the upper hierarchy is supplied to the lower hierarchy as an average pixel value of the block in a lower hierarchy located immediately below the upper hierarchy. The described image processing program. The image processing program is an encoding program for compressing an image,
Calculating a difference between a predicted pixel value of the sub-block and a true pixel value as a prediction residual;
Performing an irreversible transformation on the calculated prediction residual;
Generating an alternating current component of an image as a part of compressed data by performing entropy coding on the prediction residual subjected to the irreversible transformation;
Generating the prediction residual to be supplied to the adder by performing inverse processing of the lossy transformation on the prediction residual subjected to the lossy transformation. The image processing program according to claim 13 or 14. The image processing program is a decoding program for expanding an image,
Restoring the prediction residual used for calculating the average pixel value of the sub-block by performing inverse processing of irreversible transformation and entropy coding performed at the time of image compression on the compressed data of the image; The image processing program according to claim 13 or 14, further comprising:

Images