JP2005094212A - Image processor and processing method, computer program, and computer readable storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology realizing high speed filtering in which blocking technique is employed in view point of memory efficiency and occurrence of discontinuity is retarded at the boundary of blocks. <P>SOLUTION: A blocking section 201 splits input image data into a plurality of image blocks. A block processing section 202 receives block image data and performs filtering. Intermediate results of filtering are stored in an intermediate result storage memory 203 and a remarked block is filtered with reference to intermediate results stored temporarily in the filtering adjacent to that remarked image block. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and method, a computer program, and a computer-readable storage medium.

  Considering processing of large image data, a large buffer memory is required if this is handled as a group, so there is a processing method by dividing an image into a plurality of blocks (Patent Document 1).

  However, when only the image data in the block is referred to when performing processing on the block, the result of the processing includes a problem that it is likely to be discontinuous at the boundary portion of the block.

  On the other hand, there is a technique called wavelet transform processing in the international standard ISO / IEC15444-1: 2000 (so-called JPEG2000). In this technique, since the mirror image generated by folding the data inside the boundary is used for the data outside the boundary, the above problem is alleviated to some extent. However, there is still room for improvement compared to the case where the block is not divided.

Therefore, it is desirable to process image data in a lump. In this case, a frame memory or a line memory for holding the image data is required. Considering that recent digital cameras have a resolution of several million pixels, a large amount of memory is required.
JP-A-5-64010

  The present invention has been made in view of such problems, and adopts a block division technique from the viewpoint of memory efficiency, makes it difficult for discontinuities to occur at the boundary between blocks, and further provides a filter. The present invention intends to provide a technique for speeding up the processing.

In order to solve this problem, for example, an image processing apparatus of the present invention has the following configuration. That is,
An image processing device that performs filtering on input image data,
Dividing means for dividing the input image data into a plurality of image blocks;
Filter means for sequentially inputting and filtering image data of each image block divided by the dividing means,
The filter means includes
Storage means for temporarily storing intermediate results in the vicinity of the periphery in the target image block when filtering the target image block;
And means for performing a filtering process with reference to an intermediate result temporarily stored in the already-filtering process adjacent to the target image block when the target image block is filtered.

  According to the present invention, an input image is divided into a plurality of blocks, and processing is performed for each block, and the filter processing is performed with a simple configuration and at high speed while maintaining the continuity of the boundary portion of the block. Becomes possible.

  Embodiments according to the present invention will be described below with reference to the accompanying drawings.

  FIG. 1 is a configuration diagram of a main part of an image processing apparatus according to an embodiment.

  The block processing means 102 processes a rectangular block image, but the size of the rectangular block is limited to a certain finite width, height, or area. The block dividing unit 101 is for dividing the input image data into sizes that can be handled by the block processing unit 102. The intermediate result storage means 103 is for temporarily storing intermediate results to be used later by the block processing means. The raster conversion means 104 is for making the output of the block processing means 101 in the same order as the corresponding pixels in the input image. If there is no problem in the subsequent processing as image data divided into blocks, the raster conversion means 104 can be omitted.

  FIG. 2 shows a configuration of an encoding device based on JPEG2000 encoding of the configuration of FIG.

  This encoding apparatus can obtain the same result as that obtained by performing wavelet transform on the entire image at the same time in the wavelet transform processing divided into blocks according to the present embodiment, and avoids deterioration in image quality due to tile division in JPEG2000. .

  The block dividing unit 201 divides image data input into blocks that can be processed by the subsequent block processing unit 202 to generate block image data.

  The block processing unit 202 inputs the block image data from the block dividing unit 201 and the intermediate result data of the process for the previous block read from the intermediate result storage memory 203, outputs the processing result to the quantization unit 204 and the subsequent The intermediate result is output to the intermediate result storage memory 203 and stored in preparation for the processing of the block to be performed.

  The quantization unit 204 and the entropy coding unit 205 are for performing quantization or entropy coding according to a method defined in JPEG2000, respectively. In JPEG2000, entropy encoding is specified to be performed in units of rectangular areas called coding blocks. Therefore, the raster conversion means 104 in FIG. 1 is not necessary in this embodiment.

  Hereinafter, the details of each part in FIG. 2 will be described.

  As shown in FIG. 3, the block division unit 201 folds back each side of the input image 301 (bold solid line), adds a partial mirror image, and generates a division result for the mirrored image 302. The procedure first creates a region 306 that is line-symmetric with respect to the boundary line 305 in the vicinity of the boundary of the input image 301. Thereafter, a process for adding mirror images to the left and right is performed in the same manner.

  Here, the column 307 will be described as an example. If the pixel values included in the column 307 are {a, b, c, d, e, f, g, ...} in order from the upper side (boundary), the pixel value of the mirror image 308 is the top pixel {a}. The removed four pixels {b, c, d, e} are arranged in reverse order {e, d, c, b} (note that the number of folded pixels is four. In this embodiment, the number of taps is This means that the case of “9” is assumed, but is not limited to this number).

  Therefore, the mirror image-attached row 309 obtained by connecting the mirror image 308 and the row 307 is {e, d, c, b, a, b, c, d, e, f, g,. It becomes.

  By doing so, there is an effect of suppressing high-frequency components at the edge of the image during two-dimensional discrete wavelet transform by the subsequent block processing unit 202, and suppressing deterioration in image quality during compression.

  Next, the mirrored image 302 is divided into sizes determined vertically and horizontally as indicated by reference numerals 303 and 304 in the drawing. As shown in the figure, depending on the size of the input image, it may be half-finished, but in this case, the blocks located at the right end and the lower end are output with smaller sizes than usual.

  In addition to each pixel data (pixel signal value) in the block, the block dividing unit 201 indicates information about the size of the block and whether the block corresponds to the end (upper, lower, left, and right) of the input image. To pass.

  The block processing unit 202 refers to the intermediate result data stored in the intermediate result storage memory 203 obtained by the processing of the previous block, and uses the intermediate result data for use in the processing of other blocks as the intermediate result storage memory. Save to 203.

  FIG. 4 shows the range of image data input to each block and the range of output image data.

  The block processing unit 202 performs processing in the order of the first block on the upper left and the second block on the right and the block immediately below the first block after reaching the right end.

  When processing the first block, the block processing unit 202 performs the input on the input range 401 (in the solid line frame) and outputs the result to the first block output range 402 (grid unit).

  First, the calculation (discrete wavelet transform) shown in FIG. As a result of this vertical (vertical) direction wavelet transform, an L (low frequency) component and an H (high frequency) component are obtained.

  Next, the results are arranged for each of the L component and the H component, and this time, the calculation shown in FIG. 7 is further performed for each row connected in the horizontal (horizontal) direction for each component. By this horizontal wavelet transform, an LL component and an HL component are obtained from the L component, and an LH component and an HH component are obtained from the H component. These LL component, HL component, LH component, and HH component are output as a two-dimensional wavelet transform result.

  As can be seen from FIG. 7, since 2n-8 outputs are obtained for 2n inputs, the output range is smaller than the input range. In other words, the mirror image added to the original image described above is for creating the left and right four pixel data.

Hereinafter, the discrete wavelet transform of FIG. 7 will be described. Assume that 2n input pixel values continuous in the vertical or horizontal direction are {X ₀ , X ₁ , X ₂ ,..., X _2n−1 } in order.

First, {X ′ ₁ , X ′ ₃ ,..., X ′ _2n-3 } is obtained according to the following equation.
X ' _{2m + 1} = X _{2m + 1} + α (X _2m + X _{2m + 2} )
Next, {X ″ ₂ , X ″ ₄ ,..., X ″ _2n-4 } is obtained according to the following formula.
X '' _2m = X _2m + β (X ' _2m-1 + X' _{2m + 1} )
Next, {X ⁽³⁾ ₃ , X ⁽³⁾ ₅ ,..., X ⁽³⁾ _2n-5 } is obtained according to the following equation.
X ⁽³⁾ _{2m + 1} = X ' _{2m + 1} + γ (X'' _2m + X'' _{2m + 2} )
Next, {X ⁽⁴⁾ ₄ , X ⁽⁴⁾ ₆ ,..., X ⁽⁴⁾ _2n-6 } is obtained according to the following equation.
X ⁽⁴⁾ _2m = X '' _2m + δ (X ⁽³⁾ _2m + X ⁽³⁾ _{2m + 2} )
Finally, low frequency components {Y ₄ , Y ₆ ,..., Y _2n-6 } and high frequency components {Y ₅ , Y _{7 ,.}
Y _2m = 1 / k ・ X ⁽⁴⁾ _2m
Y _{2m + 1} = k ・ X ⁽³⁾ _{2m + 1}
As described above, n-4 pieces of low frequency component data and n-4 pieces of high frequency component data are obtained from _2n input pixel values {X ₀ , X ₁ , X ₂ ,..., X _2n-1 }. Generated and separated from each other.

  The filtering process is performed in the vertical direction and the horizontal direction, and data in the illustrated lattice region 402 is output. Note that the output range of the first block processing is the illustrated grid area 401, data included in the first block in the hatched area 404, data included in the first block in the hatched area 406, and Does not output data included in the first block in the vertical line area 408. The reason is that the data of the region not to be output is a region where the wavelet transform is not completed. However, since the processing up to the intermediate stage of the above-described filter processing is performed on a part of each of the oblique line region 404, the oblique lattice region 406, and the vertical line region 408 existing in the first block, Data shall be retained.

  In the processing of the second block, the second block output range 404 (shaded portion) is output in response to the input of the second block input range 403 (within the solid line frame). The output spread to the left side beyond the input range is a portion where the output is determined by the intermediate result for the first block input range 401 and the input of the second block input range 403 in the first block.

In the processing of the second block, what is required as an intermediate result is the value indicated by reference numeral 701 in FIG. 7 in the horizontal wavelet transform processing of the first block, that is, {X _2n−1 , X _2n−2 , X ′ _2n-3 , X '' _2n-4 , X ⁽³⁾ _2n-5 }.

  In the present embodiment, at the time of the horizontal wavelet transform processing of each row of the first block, these intermediate results are stored in the intermediate result storage memory 203, and when performing the wavelet transform in the second block, Intermediate data is used.

FIG. 8 is a diagram illustrating a wavelet transform process using the intermediate result 801. Intermediate result 801 {X ₋₁ , X ₋₂ , X ′ ₋₃ , X ″ ₋₄ , X ⁽³⁾ ₋₅ } of the adjacent block and n input values {X ₀ , X ₁ , X ₂ , ..., X _n-1 }, n output values {Y _-4 , Y _-3 , Y _-2 , ..., Y _n-5 } and intermediate results 802 {X _n-1 used in the subsequent block , X _n−2 , X ′ _n−3 , X ″ _n−4 , X ⁽³⁾ _{n− 5} }.

As another method, the operation of the part whose value is already known is executed first, and the value indicated by reference numeral 702, that is, {X _2n-1 + X _2n-2 , X _2n-2 + βX ' _{2n −3} , X ′ _2n−3 + γX ″ _2n−4 , X ″ _2n−4 + δX ⁽³⁾ _2n−5 } can be an intermediate result.

  Thus, since the filter used in this example has a maximum of 9 taps, it is necessary to superimpose 8 pixels in the conventional method, and it is necessary to hold data for 8 pixels for each row or column. According to this embodiment, 4 to 5 intermediate results are sufficient.

  In the processing of the fifth block, the fifth block output range 406 (hatched grid portion) is output with respect to the fifth block input range 405 (within the solid line frame). In the process of the fifth block, by receiving the intermediate result of the vertical wavelet transform process of the first block, the output range is expanded upward with respect to the input range.

  In the processing of the sixth block, the sixth block output range 408 (vertical line portion) is output with respect to the sixth block input range 407 (within the solid line frame). Similarly to the fifth block, the sixth block process receives the intermediate result of the vertical wavelet transform process in the second block. Furthermore, by receiving the intermediate result of the horizontal wavelet transform in the fifth block, the output range is expanded upward and left with respect to the input range.

  Overall, the image data output of all blocks connected is smaller than the image data input of all blocks connected. This is because the part to which the mirror image is added is deleted by the block dividing unit 201 and returns to the same size as the first input image data.

  FIG. 6 is a diagram illustrating a data flow between the block processing unit 601 and the intermediate result storage memory 602. The intermediate result storage memory 602 stores two types, a horizontal wavelet transform intermediate result 603 that propagates between blocks adjacent to the left and right, and a vertical wavelet transform intermediate result 604 that propagates between blocks adjacent vertically. .

  For the vertical wavelet transform intermediate result 604, the intermediate result of the upper left block, the upper block, and the left block is referred to the block of interest (the upper block is only the upper block in the left end block). Therefore, in the case of block division as shown in FIG. 3, it is only necessary to refer to the process up to four blocks before the target block, and intermediate data before that is unnecessary. In other words, the configuration of the ring buffer is simplified so that the results up to four blocks before can be held and extracted in the oldest order.

  To summarize the above, the block dividing unit 201 adds a mirror image corresponding to the number of taps when performing filter processing (wavelet transform) to the periphery of input image data, and then converts the mirrored image into a plurality of images. Divide into blocks. Then, the position information of the divided block (information indicating if it is at the position of the top, bottom, left and right ends, information indicating that if it is at an intermediate position other than the top, bottom, left and right ends) and the inside of the block Are output to the block processing unit 202. Note that 4 bits are sufficient for position information, and it is sufficient to assign bits to each of the upper, lower, left, and right. Therefore, if it is the upper end and the left end, the corresponding two bits should be set to “1”, and if it is a block at an intermediate position excluding the end position, all bits should be set to “0”. It ’s fine.

  The block processing unit 202 performs wavelet transform based on the input pixel data in the block and the position information. At this time, based on the position information, which intermediate calculation result in the already processed block is determined. The reference is determined, and wavelet transform is performed based on the intermediate calculation result of the determined position. Further, based on the position information of the target block, it is determined which intermediate calculation result of the target block is to be stored in the intermediate result storage memory 203 when the target block is wavelet transformed, and processing is performed in accordance with the determination result.

  Then, the transform coefficient obtained by the wavelet transform process in the block processing unit 202 is output to the quantization unit 204, subjected to the quantization process, and the entropy coding unit 205 performs the entropy coding.

  Since the block division unit 201, the quantization unit 204, and the entropy encoding unit 205 in the above processing will not be described, the processing of the block processing unit 601 in the embodiment will be described below according to the flowchart of FIG. To do.

  FIG. 5 shows image processing in the block processing unit 601 in the embodiment, which is composed of color conversion processing, vertical wavelet conversion processing, and horizontal wavelet conversion processing, and exchanges data with the intermediate result storage memory 602. It is added.

  First, in step S501, block image data and position information are input from the block dividing unit 201. For example, in the process of the first block, the part of the first block input range 401 shown in FIG. 4 is input, and in the process of the next block, the part of the second block input range 403 is input. I can judge.

  Next, in step S502, conversion from the RGB color space to the YCbCr color space is performed according to the following equation. This is because the use of a color space suitable for human visual characteristics prevents the image quality deterioration from becoming noticeable in the subsequent compression process in the quantization unit and the entropy coding unit. Incidentally, the RGB → YCbCr conversion is obtained by the following equation.

In step S503, the vertical wavelet transform intermediate result 604 stored in the intermediate result storage memory 602 is acquired, and the process in the next step S504 is performed together with the color conversion result in step S502.
However, in the block located at the upper end of the input image, the first eight input values (color conversion result of 402) {X ₀ , X ₁ , X ₂ ,..., X ₇ } for each column as shown in FIG. Is used to calculate a portion {X ₇ , X ₆ , X ′ ₅ , X ″ ₄ , X ⁽³⁾ ₃ } indicated by reference numeral 703, which is used as an intermediate result.

  In step S504, for each color component and each column, as shown in FIG. 8, from the intermediate result and the color conversion result in step S502, the longitudinal wavelet transform result divided into the low frequency component and the high frequency component, and the subsequent Calculate the intermediate wavelet transform intermediate result used in the block.

  In step S 505, the vertical wavelet transform intermediate result used in the subsequent block is stored in the intermediate result storage memory 602. However, since it is not referred to in the block located at the lower end of the input image, it is not necessary to save it.

  Next, in step S506, the horizontal wavelet transform intermediate result 603 stored in the intermediate result storage memory 602 is acquired, and the process of the next step S507 is performed together with the vertical wavelet transform result in step S504.

However, in the block located at the left end of the input image, as in step S503, the first eight {X ₀ , X ₁ , X ₂ ,..., X ₇ } are used to calculate a portion {X ₇ , X ₆ , X ′ ₅ , X ″ ₄ , X ⁽³⁾ ₃ } indicated by reference numeral 703, which is used as an intermediate result.

  In step S507, for each of the low-frequency component and high-frequency component divided in step S504, for each color component, for each row, as shown in FIG. Calculate the intermediate wavelet transform intermediate result used in the block.

  In step S508, the intermediate wavelet transform intermediate result used in the subsequent block is stored in the intermediate result storage memory 602. However, since the block located at the right end of the input image is not referred to, it need not be saved.

  In step S509, a two-dimensional wavelet transform result including an LL component, an HL component, an LH component, and an HH component is output to the quantization unit 204 as a result of image processing for the input in step S501.

  By processing each block as described above, wavelet transform is performed on the entire image without redundant calculations due to overlapping of the blocks, and by storing fewer intermediate results compared to the overlapped portion. It is possible to obtain the same result as the case. Therefore, it is possible to handle the whole as one tile while performing processing in block units, and it is possible to realize a JPEG2000 encoding apparatus that avoids deterioration in image quality due to tile division.

<Description of modification>
FIG. 9 is a block diagram of a modification of the example of FIG. 1, and corresponds to an example in which a plurality of portions corresponding to the block processing unit 202 in FIG. 2 are provided.

  The large block processing means 902 performs a plurality of small block processes for performing the process of determining the output value using the intermediate result of the same already processed block and the input value of the new block as described above. Means (905a, 905b,...) And intermediate result transmission means 903.

  The small block processing means 905a, 905b... Process a rectangular block image, but the size of the rectangular block is limited to a certain finite width, height, or area. The small block processing means 905a and 905b are connected via the intermediate result transmission means 903 so that the result of processing the previously described block can be used in the processing of the adjacent block.

  The intermediate result transmission unit 903 temporarily stores an intermediate result generated by one small block processing unit for use by another small block processing unit.

  The large block processing unit 902 can process a block larger than a block that can be handled by one small block processing unit by a plurality of small block processing units 905a, 905b,. Yes.

  The block dividing unit 901 divides input image data into sizes that can be handled by the small block processing units 905a and 905b and supplies them in order.

  The raster conversion means 904 is for making the output of the large block processing means 902 in the same order as the corresponding pixels in the input image. If there is no problem with the image data divided into blocks, the raster conversion means 904 can be omitted.

  FIG. 10 is a block diagram of a specific apparatus. Assume that the resolution (number of pixels) of the input image data is doubled in both the main scanning direction and the sub-scanning direction with respect to the existing image processing apparatus example of FIG. Here, the processing performance of the block processing unit 202 in the processing apparatus of FIG. 2 is improved.

  The apparatus shown in FIG. 10 is greatly different from FIG. 2 in the following two points.

  First point: As described later with reference to FIG. 11, the block processing unit 1003 has four small block processing units, which are connected via an intermediate result transmission unit.

  Therefore, the block processing unit 1003 serves as both the block processing unit 202 and the intermediate result storage memory 203 in FIG.

  Second point: the small block dividing unit 1002 is provided between the large block dividing unit 1001 corresponding to the block dividing unit 201 and the block processing unit 1003. This is for further subdividing the large block divided by the large block dividing unit into small blocks that can be handled by the small block processing unit in the block processing unit 1003.

  The large block dividing unit 1001 functions in the same manner as the block dividing unit 201 in FIG. However, since the resolution is doubled, the number of pixels of the large block resulting from the division is doubled both horizontally and vertically (area = 4 times the number of pixels) compared to the block in the example of FIG. ing.

  As shown in FIG. 3, the large block dividing unit 1001 folds back each side of the input image 301 (thick solid line), adds a mirror image, and outputs the division result for the mirrored image 302 (broken line). In this procedure, first, a hatched portion 306 that is a mirror image is created for the lattice portion 305 of the input image 301. Thereafter, a process for adding mirror images to the left and right is performed in the same manner. The mirror image generation process is the same as the process described above, and a description thereof is omitted here.

  Next, the mirrored image 302 is divided into sizes determined vertically and horizontally as indicated by reference numerals 303 and 304 in the drawing. As shown in the figure, depending on the size of the input image, it may be half-finished. In this case, the large blocks located at the right end and the lower end are output with smaller sizes than usual.

  The large block dividing unit 1001 divides the size of the large block in addition to each pixel signal value in the large block and the position information indicating whether the large block corresponds to the end (upper and lower left and right) of the input image. Part 1002.

  The small block dividing unit 1002 further divides the large block divided by the large block dividing unit 1001 into four small blocks so that each small block can be handled by the block processing unit 203 in FIG. Each pixel signal value in the small block, the size of the small block, and the position of the small block in the large block (any one of the upper left, upper right, lower left, and lower right) are passed to the block processing unit 1003.

  Since the processing order in the block processing unit 1003 is performed in the order of upper left, upper right, lower left, and lower right, the small block dividing unit 1002 outputs the small blocks divided in the same order.

  The small block dividing unit 1002 also passes edge information to the block processing unit 1003 for the small block including the edge of the input image. The edge information determines whether or not each of the upper edge, the lower edge, the right edge, and the left edge corresponds to the edge of the input image. The edge information from the large block dividing unit 1001 and the small block It can be determined by the position in the large block.

  The block processing unit 1003 is for performing the same two-dimensional discrete wavelet transform as the block processing unit 203 in FIG. Details will be described later with reference to FIG.

  The quantization unit 1004 and the entropy encoding unit 1005 perform quantization and entropy encoding according to the method defined in JPEG2000, similarly to the quantization unit 204 and the entropy encoding unit 205 in FIG. Is for.

  FIG. 11 is a diagram showing details of the block processing unit 1003 in FIG.

  The input distribution unit 1101 assigns data of one small block from the small block division unit to one of the four small block processing units 1103, 1104, 1105, and 1106.

  Specifically, in the large block, the upper left small block is the first small block processing unit 1103, the upper right is the second small block processing unit 1104, the lower left is the third small block processing unit 1105, and the lower right is the fourth small block processing. The unit 1106 takes charge of the processing in the above order.

  The processing contents of the small block processing units 1103, 1104, 1105, and 1106 are as shown in FIG. 5 and are the same as those of the block processing unit 202 in FIG. Is possible.

  Each small block processing unit 1103, 1104, 1105, 1106 is also connected via intermediate result transmission units 1107, 1108,..., 1114 as shown in the figure, and transmits the intermediate result to other small block processing units. It is possible.

  The first intermediate result transmission unit 1107, the third intermediate result transmission unit 1109, the fifth intermediate result transmission unit 1111 and the seventh intermediate result transmission unit 1113 each transmit a horizontal wavelet transform intermediate result corresponding to one small block. To do.

  The second intermediate result transmission unit 1108 and the fourth intermediate result transmission unit 1110 each transmit a vertical wavelet transform intermediate result corresponding to one small block.

  The sixth intermediate result transmission unit 1112 and the eighth intermediate result transmission unit 1114 have the same number of vertical wavelets as the number of large blocks satisfying the width of the input image as described with reference to FIG. 6 (four in the example of FIG. 3). By holding the conversion intermediate result, it becomes possible to transmit the intermediate result between blocks adjacent vertically.

  The first small block processing unit 1103 includes a third intermediate result transmission unit 1109 (except when the processing target small block is at the left end) and a sixth intermediate result transmission unit 1112 (except when the processing target small block is at the top). The intermediate result is received, the output value is determined based on the pixel value in the processing target small block and the received intermediate result, and the vertical intermediate result is transmitted to the first intermediate result transmission unit 1107, and the horizontal intermediate result is transmitted to the second intermediate result. Stored in the unit 1108.

  The first small block processing unit 1103 receives the intermediate result from the sixth intermediate result transmission unit 1112, determines the output value based on the pixel value in the processing target small block and the received intermediate result, and also displays the vertical intermediate result. The first intermediate result transmission unit 1107 and the horizontal intermediate result are stored in the second intermediate result transmission unit 1108.

  The second small block processing unit 1104 receives the intermediate results from the first intermediate result transmission unit 1107 and the eighth intermediate result transmission unit 1114 (except when the processing target small block is at the upper end), and receives the intermediate result in the processing target small block. The output value is determined based on the pixel value and the received intermediate result, and the vertical intermediate result is stored in the third intermediate result transmission unit 1109 and the horizontal intermediate result is stored in the fourth intermediate result transmission unit 1110.

  The third small block processing unit 1105 receives the intermediate results from the seventh intermediate result transmission unit 1113 (except when the processing target small block is at the left end) and the second intermediate result transmission unit 11108, and receives the intermediate result in the processing target small block. The output value is determined based on the pixel value and the received intermediate result, and the vertical intermediate result is stored in the fifth intermediate result transmission unit 1111 and the horizontal intermediate result is stored in the sixth intermediate result transmission unit 1112.

  The fourth small block processing unit 1106 receives intermediate results from the fifth intermediate result transmission unit 1111 and the fourth intermediate result transmission unit 1110, and determines an output value based on the pixel value in the processing target small block and the received intermediate result. At the same time, the horizontal intermediate result is stored in the seventh intermediate result transmission unit 1113 and the vertical intermediate result is stored in the eighth intermediate result transmission unit 1114.

  As can be seen from the figure, the second small block processing unit 1104 and the third small block processing unit 1105 do not affect each other's results, so that the processes can be performed in parallel. Also, the fourth small block processing unit 1105 and the first small block processing unit 1103 at the stage of processing the next large block can be operated in parallel in the same manner.

  The processing time can be shortened by operating in parallel as described above.

  The output organizing unit 1102 is provided to smoothly transmit the outputs of the small block processing units 1103, 1104, 1105, and 1106 operating in parallel as described above to the quantization unit.

  As described above, according to the present modification, a portion corresponding to the block processing unit 202 in the first embodiment is divided into a plurality of small block processing units (four in the above example, 2 × 2) and between them. By comprising the intervening transmission unit, the processing speed of the entire block processing unit can be further increased.

<Second Embodiment>
Hereinafter, as a second embodiment, resolution conversion processing by the bilinear method will be described.

  When enlarging from A5 size to A4 size, resolution conversion is about 1: 1.41. In such a case, there are, for example, a bilinear method and a bicubic method as a method for determining a pixel value at an odd position. Hereinafter, an example of the bilinear method will be described.

  Conversion from A5 size to A4 size is multiplied by 1.41, in other words, if the resolution is the same, this is equivalent to increasing the number of horizontal pixels and the number of vertical pixels by 1.41 times. At this time, assuming that each pixel data of the original image is X (i, j) at the coordinate position and pixel data after enlargement (pixel data to be calculated) is P, as shown in FIG. Of four original pixels D (= X (i, j)), C (= X (i + 1, j)), A (= X (i, j + 1)), B (= X (i + 1, j + 1)) It will be in the rectangle.

Therefore, in the bilinear method, the following formula is calculated by the weight according to the distance between these four points (four original pixels) A, B, C, and D (between two adjacent pixels in the horizontal or vertical direction). The distance is “1”).
P = (1-dx) x (1-dy) x A + dx x (1-dy) x B + dx x dy x C + (1-dx) x dy x D
When trying to apply such a method to an image divided into blocks, it is necessary to refer to pixel values belonging to adjacent blocks in the vicinity of the block boundary.

  As shown in FIG. 13, the entire image 1301 to be subjected to resolution conversion is divided into 16 blocks of 4 × 4. It is assumed that processing is performed in order from the upper left, like a first block 1002, a second block 1003,.

  Each point in the first block 1302 indicates a sampling position in the original image.

  An image 1304 after resolution conversion is a diagram showing sampling positions when resolution conversion is performed on the first block 1302 and the second block 1303. In the figure, the “x” mark is the sampling position of the converted image. In the range indicated by reference numeral 1305 in the figure, the point of the original image to be referenced extends not only in the first block but also in the second block.

  For the points corresponding to such sampling positions, in the processing of the first block, partial results are obtained by using only the points of the original image (for example, points A and D in FIG. 12) belonging to the first block. calculate.

  Then, the position and the partial result based on the original pixel value in the block are stored as intermediate results for use in processing of adjacent blocks.

  In the subsequent block processing, the partial result based on the current pixel value in the block is calculated and summed with the stored result, so that interpolation by the bilinear method can be realized even near the block boundary.

  When using a plurality of image processing means for performing the processing as described above, parallel operation becomes possible by devising the order of processing within the block as described below.

  That is, processing of a part in which an intermediate result of a peripheral part (for example, reference numeral 1305 in the first block) that may affect an adjacent block is first calculated, and then a result can be obtained only from pixel values in the block And finally, processing of a portion (for example, reference numeral 1305 in the second block) that needs to refer to an intermediate result from an adjacent block is performed.

<Third Embodiment>
Next, a synthesis process of an original image and a certain repetitive pattern will be described as a third embodiment.

  FIG. 14 is a diagram for explaining the composition process.

  In this process, the original image is output as it is in the portion marked “◯” in the repetitive pattern 1401, and the pixel value of the original image is replaced with a predetermined value in the portion marked “x”. Then, as shown in the figure, an object is to obtain an effect that a repetitive pattern 1401 is repeated without a gap, and the entire processing target image 1402 is shaded with a specific pattern.

  At this time, when the block boundary (solid line) does not coincide with the pattern boundary (broken line) as shown in the figure, it is necessary to devise a technique for preventing the pattern from becoming discontinuous. For example, in the processing of each block, there is a method for obtaining a value obtained by dividing the distance from the origin by the width and height of the pattern.

  However, division is not preferable because the circuit is more complicated than other operations and it takes time. In this embodiment, a counter for the number of rows and the number of columns is used in the processing of each block, and when the count value reaches the size of the repeated pattern 1401, which position on the repeated pattern 1401 should be referred to is reset. Shall be managed.

  At this time, assuming that processing is performed in order from the upper left block, the number of columns used for the right adjacent block and the number of used rows for the lower adjacent block are transmitted as intermediate results. By using the received intermediate result value as the initial value of the counter, it becomes possible to perform continuous processing on the entire image.

  Although each embodiment has been described above, the present invention is not limited to the above embodiment, and various applications are possible.

  For example, even in a general-purpose information device such as a personal computer, wavelet transformation of image data can be performed at high speed if processing is performed according to the procedure shown in FIG. Therefore, it is clear that the present invention also includes such a computer program. Usually, the computer program is stored in a portable computer-readable storage medium such as a floppy (R) disk or CDROM, and can be executed by setting the medium in a computer and copying or installing it. Therefore, it is clear that a computer-readable storage medium is also included in the scope of the present invention.

It is a basic lineblock diagram of an embodiment. It is a block block diagram of the apparatus in 1st Embodiment which embodied FIG. It is a figure which shows a mirror image process and block division. It is a figure which shows the relationship between the area | region of input and output in each block process. It is a flowchart which shows the processing content of a block process part. It is a figure for demonstrating the movement of the data between a block process part and an intermediate result storage memory. It is a figure for demonstrating the calculation of wavelet transformation. It is a figure for demonstrating the wavelet transformation process using an intermediate result. It is a block block diagram of a modification. FIG. 10 is a block diagram of an apparatus embodying FIG. 9. It is a figure which shows the detail of the block process part in FIG. It is a figure for demonstrating a bilinear method. It is a figure for demonstrating the interpolation process in 2nd Embodiment. It is a figure for demonstrating the image processing in 3rd Embodiment.

Claims (8)

An image processing device that performs filtering on input image data,
Dividing means for dividing the input image data into a plurality of image blocks;
Filter means for sequentially inputting and filtering image data of each image block divided by the dividing means,
The filter means includes
Storage means for temporarily storing intermediate results in the vicinity of the periphery in the target image block when filtering the target image block;
An image processing apparatus comprising: means for performing a filtering process with reference to an intermediate result temporarily stored in an already-filtering process adjacent to the target image block when the target image block is filtered. The filter processing means is means for performing a filter operation to be performed in the horizontal and vertical directions of input image data using a one-dimensional filter having a predetermined number of taps for performing wavelet transform,
The image processing apparatus according to claim 1, wherein the storage holding unit stores and holds an intermediate calculation result at each stage when the one-dimensional filter processing of the tap is performed. The filter processing means includes
Means for further dividing the image block into a plurality of small blocks;
Small block filter means for performing filtering on each of the divided small blocks;
The image processing apparatus according to claim 1, further comprising: a transmission unit that transmits intermediate data that is being filtered by each small block filter unit.   The image processing apparatus according to claim 1, wherein the filter processing unit is a synthesis process of an input image and a predetermined repetition pattern.   The image processing apparatus according to claim 1, wherein the filter processing unit is a unit that generates an interpolation pixel from the input image in order to change the resolution or the number of pixels of the input image. An image processing method for filtering input image data,
A dividing step of dividing the input image data into a plurality of image blocks;
A filter step of sequentially inputting and filtering image data of each image block divided in the division step,
The filtering step includes
A storage step of temporarily storing intermediate results in the vicinity of the periphery of the target image block when filtering the target image block;
And a step of performing a filtering process with reference to an intermediate result temporarily stored in the already-filtering process adjacent to the target image block when the target image block is filtered. A computer program that functions as an image processing device that filters input image data by being read and executed by a computer,
Dividing means for dividing the input image data into a plurality of image blocks;
Filter means for sequentially inputting and filtering image data of each image block divided by the dividing means,
The filter means includes
Storage means for temporarily storing intermediate results in the vicinity of the periphery in the target image block when filtering the target image block;
A computer program that functions as means for performing filtering processing with reference to an intermediate result temporarily stored in filter processing that is adjacent to a target image block when filtering the target image block.   A computer-readable storage medium storing the computer program according to claim 7.

Images