JP6324174B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method for rotating an image by burst-reading image data from a memory.

  In recent years, an arbitrary angle rotation process for rotating an input image at an arbitrary angle in an image processing apparatus may be required. This arbitrary angle rotation processing is used for correction processing (skew correction) in which, for example, when the document is tilted during scanning, the tilt of the scanned document is measured and the tilt is canceled. If the coordinates of the image before rotation are (x, y) and the coordinates of the image after rotation are (X, Y), the relationship between them is given by the following equation (1) using the rotation angle θ. However, since the coordinate (x, y) corresponding to the case where the coordinate (X, Y) is the center of the pixel is not necessarily the center of the pixel, it is generally calculated by interpolation from surrounding pixels.

  Conventionally, image data input to an image processing apparatus is continuously stored for each row on a memory in the apparatus. FIG. 1A is a diagram illustrating a state in which images are continuously stored for each row in the memory. In FIG. 1A, an image 100 is an image having a width of 16 pixels, and the numbers 0, 1, 2,... Each indicate an address (hexadecimal number; the same applies hereinafter) of each pixel stored in the memory. ing. For example, the pixel at coordinate (0,0) is stored at address 0, the pixel at coordinate (1,0) is stored at address 1, and the coordinate (3,2) is stored at address 23. Assume that image rotation processing at a rotation angle θ is performed on the input image 100 stored in this way (see FIG. 1B). At this time, one line in the output image is located at the arrow in the figure. For this reason, the pixel value of the rotated image is calculated from the neighboring pixels by using an interpolation operation, and for example, a pixel (processing target pixel) of a portion indicated by a halftone dot in the drawing is required.

  When reading the pixel to be processed from the memory, high-speed reading by burst access is possible for continuous areas. Here, assuming that the burst length is 8 pixels, in order to read out all the pixels in the halftone dot portion in FIG. 1B, it is necessary to read out 1 × 8 pixels 8 times (see FIG. 1C). ). In this case, a total of 64 pixels are read out, but the necessary halftone dot pixels (processing target pixels) are 36 pixels, which is inefficient. As described above, if the image data stored in the memory for each row is accessed in an oblique direction, the efficiency of burst access cannot be improved and reading takes time.

  In this regard, for example, in the method described in Patent Document 1, burst reading at 90 ° or 270 ° rotation is made efficient by writing image data in the memory in units of N × N pixels.

  In the method described in Patent Document 2, the size and shape of the block are selected according to the rotation angle, and the relative position between the blocks is shifted and written in the memory so that the angle is corrected.

JP 2011-188050 A JP 2006-203710 A

  However, the method described in Patent Document 1 can be applied only to two types of rotation angles of 90 ° and 270 °, and cannot cope with an arbitrary angle.

  Further, in the method described in Patent Document 2, since the shapes of the blocks are almost square, the shape is not suitable for burst access at all angles.

  In particular, in a scanner, generally, after performing shading correction, skew correction is performed, and then scanned image data is processed in a flow of image processing such as filter processing. Under such a processing order, when the scanning direction of the image is matched with the shading correction direction (sub-scanning direction), if the subsequent skew correction is performed in that order, the scanning direction changes depending on the skew angle. End up. This will reduce system performance.

  An image processing apparatus according to the present invention includes a unit that performs first image processing on an input image in a predetermined scanning order, and a memory that stores the image subjected to the first image processing in a predetermined storage unit. Means for performing second image processing including rotation processing at an arbitrary angle on an image read out in burst from the memory in the predetermined storage unit; and the predetermined storage unit at the arbitrary angle. And a determination means for determining the response accordingly.

  According to the present invention, burst reading at the time of arbitrary angle rotation processing can be performed efficiently.

It is a figure explaining the outline | summary of the conventional image rotation process. 1 is a block diagram illustrating an internal configuration of an image processing apparatus that realizes scanner correction processing according to Embodiment 1. FIG. It is a figure explaining a band perpendicular order. It is a figure explaining the positional relationship of an input image and a sensor. It is a flowchart which shows the flow of the shading correction process for 1 band. It is a figure which shows the example of a storage of the image in case the burst length of a memory is 16 pixels. FIG. 6 is a diagram illustrating a processing order of tiles in the first embodiment. It is a block diagram which shows the internal structure of a skew correction part. It is a figure which shows an example of the writing which shifted the position by a 2x8 tile. FIG. 6 is a block diagram illustrating an internal configuration of an image processing apparatus according to a second embodiment. It is a figure explaining a tile horizontal order.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In addition, the structure shown in the following Examples is only an example, and this invention is not limited to the structure shown in figure.

  In this embodiment, a mode in which a series of scanner correction processes including skew correction is performed on an image read by a scanner (hereinafter referred to as a scanned image) will be described.

  FIG. 2 is a block diagram illustrating an internal configuration of an image processing apparatus that realizes scanner correction processing according to the present embodiment. Scanner correction processing can be broadly divided into processing in the preceding image correction unit 200 and processing in the subsequent image correction unit 210. The image data input to the pre-stage image correction unit 200 is raw data read by the line sensor. The processing result of the upstream image correction unit 200 is input to the downstream image correction unit 210 via the memory 220. The processing result of the post-stage image correction unit 210 is output to the outside as a corrected image (output image). Further, the skew detection unit 230 is constituted by a dedicated sensor, for example, and measures the inclination (carrying-in angle) θ of the document set on the scanner (not shown). The measured angle θ is sent as a parameter to a reorder unit 203 and a skew feeding correction unit 211 described later.

<Pre-stage image correction unit>
First, the pre-stage image correction unit 200 will be described in detail. The pre-stage image correction unit 200 includes an HV conversion unit 201, a shading correction unit 202, and a reorder unit 203 (see FIG. 2).

  The HV conversion unit 201 performs processing for converting the input image into an image in a predetermined scanning order. In this embodiment, the input image is processed for each of a plurality of areas divided in the sub-scanning direction. Hereinafter, this divided area is referred to as a “band”. As shown in FIG. 3, in each band of the input image 300, scanning in the sub-scanning direction is repeated in the main scanning direction, and finally the entire input image 300 is scanned. Hereinafter, the scanning order shown in FIG. 3 will be referred to as a band vertical order. As will be described later, this system-specific scanning order is used for the purpose of improving shading correction efficiency. The image data converted into an image in the band vertical order by the HV conversion unit 201 is sent to the shading correction unit 202.

  The shading correction unit 202 performs a process of correcting the gain for each sensor. FIG. 4 is a diagram illustrating the positional relationship between the input image 300 and the sensor. In a general scanner, the sensor (line sensor) 400 has sensor pixels arranged in the main scanning direction. Therefore, one column in the band vertical order image is a pixel imaged by the same sensor pixel. For this reason, in the band vertical order, parameters necessary for shading correction need only be updated for each column, and processing can be performed efficiently.

  FIG. 5 is a flowchart showing the flow of shading correction processing for one band.

  In step 501, the shading correction unit 202 acquires a shading parameter that is a parameter corresponding to the gain of each sensor pixel.

  In step 502, the shading correction unit 202 determines a target pixel in the processing target column in the band to be processed. At this time, the target pixel is determined in accordance with the band vertical order described above.

  In step 503, the shading correction unit 202 performs a shading correction process on the determined pixel of interest using the shading parameters acquired in step 501.

  In step 504, the shading correction unit 202 determines whether the target pixel is the last pixel in the processing target column. If the target pixel is the last pixel in the processing target column, the process proceeds to step 505. On the other hand, if the target pixel is not the last pixel in the processing target column, the process returns to step 502, the next pixel is determined as the target pixel, and the processing is continued.

  In step 505, the shading correction unit 202 determines whether the target pixel is the last pixel in the band to be processed. If the target pixel is not the last pixel in the processing target band, the process returns to step 501 to acquire the shading parameter of the next processing target column, and the processing for the next processing target column is continued. On the other hand, if the target pixel is the last pixel in the band to be processed, this process ends.

  The above is the content of the shading correction processing for one band, and such processing is executed for all bands. Thus, by setting the scanning order to the band vertical order, the same shading parameter can be used for each column. For example, when the band is scanned in the horizontal direction, it is necessary to acquire a shading parameter for each pixel. However, in this embodiment, this is not necessary, and the frequency of acquiring the shading parameter can be reduced. The image data subjected to the shading correction is sent to the reorder unit 203.

  The reorder unit 203 rearranges the pixel rows of the image data subjected to the shading correction into a shape suitable for the processing of the subsequent skew correction unit 211 according to the angle θ detected by the skew detection unit 230. Write to the memory 220. The reorder unit 203 has a temporary memory capable of accumulating a plurality of band heights therein, and packs pixels for each predetermined storage unit using the temporary memory. FIG. 6 is a diagram illustrating an example of storing an image in the memory 220 when the burst length of the memory 220 is 16 pixels. Fig. 6 (a) shows the case where the tile shape is 1x16, (b) shows the case where the tile shape is 2x8, (c) shows the case where the tile shape is 4x4, and (d) shows the case where the tile shape is 8x2. (E) corresponds to the case where the tile shape is 16 × 1. One tile of any selected tile shape is packed in the memory 220 so that it can be burst-accessed. For example, when an 8x2 tile shape is selected, 8 rows by 2 columns 16 pixels from coordinates (0,0) to coordinates (7,1) are stored from address 0 to address f in memory 220. Become. Here, a tile shape selection method will be described.

  The tile shape may be selected such that the angle formed by the diagonal line of the tile and the scanning direction (sub-scanning direction in the band vertical order) is closest to the detected angle (rotation angle) θ. In the case of each tile shape of 1x16, 2x8, 4x4, 8x2, and 16x1 shown in (a) to (e) of FIG. , Arctan (1), arctan (1/4), arctan (1/16). For example, if the rotation angle θ is around 45 °, a 4 × 4 tile shape may be selected. At the time of selection, the angle formed between the diagonal line of the tile and the scanning direction may be obtained based on the rotation angle θ, and the most appropriate tile shape may be selected. Alternatively, a table that defines the correspondence between the rotation angle θ and the tile shape may be prepared in advance, and the tile shape corresponding to the detected angle θ may be selected with reference to the table. By determining the storage unit in the memory according to the detected angle θ, the number of pixels for one scan that can be included in one tile can be maximized, and the efficiency of burst access is increased. be able to.

  The choice of tile shape can be a combination of natural numbers P and Q (vertical P pixel, horizontal Q pixel tile shape) satisfying N = PxQ, where the burst length for accessing the memory 220 is N pixels. That's fine. For example, if N = 12, the candidate tile shapes to be selected are 1 × 12, 2 × 6, 3 × 4, 4 × 3, 6 × 2, and 12 × 1. By setting the tile shape to a rectangular region of such vertical P pixels and horizontal Q pixels, it is possible to store all the images in the memory 220, and the memory address can be obtained by a simple calculation.

<Second-stage image correction unit>
Next, the post-stage image correction unit 210 will be described in detail.

  The post-stage image correction unit 210 includes a skew correction unit 211 and a filter processing unit 212 (see FIG. 2).

  The image (target image) stored in the memory 220 by the reorder unit 203 is burst read in the storage unit at the time of writing and sent to the skew correction unit 211. Here, taking a case where 8 × 2 is selected as the tile shape as an example, a state in which the image data stored in the memory 220 is sent to the skew feeding correction unit 211 will be described. FIG. 7A is a diagram showing a state in which one band in the band vertical order is rotated by θ, and FIG. 7B shows the order when burst reading is performed in storage units by numbers 1 to 39. ing. Since the output from the skew correction unit 211 is in this order, the input to the skew correction unit 211 is read from the memory 220 in storage units (here, 8 × 2 tiles) according to the order shown in FIG. 7B. It will be. Since the angle θ can be changed for each page, this order can also be changed for each page.

FIG. 8 is a block diagram illustrating an internal configuration of the skew feeding correction unit 211. The 8 × 2 tile data input to the skew correction unit 211 is first stored in the buffer 810. The buffer 810 can hold a plurality of 8 × 2 tile data, and holds all the pixels until they are not referred to in the current band (pixels that are no longer referred to in the current band are discarded at any time). Next, a predetermined reference pixel group necessary for interpolation is extracted from the buffer 810 and sent to the interpolation calculation unit 820. The predetermined reference pixel group is, for example, a 2 × 2 pixel group in the vicinity of the processing target pixel if the calculation in the interpolation calculation unit 820 is linear interpolation. The coordinate calculation unit 830 updates the coordinates of the processing target pixel in accordance with the band vertical order, and calculates the corresponding coordinates before rotation using the angle θ. As a result, the coordinates of the reference pixel group necessary for the interpolation operation and the coordinates of the non-reference pixels that are no longer referred to in the current band are sent to the buffer 810. Then, an interpolation calculation parameter (phase) is sent to the interpolation calculation unit 820. Further, the memory address of the 8 × 2 tile to be read next is sent to the memory 220. In this way, the interpolation calculation result (pixel data subjected to rotation processing according to the angle θ) in the interpolation calculation unit 820 is output to the filter processing unit 212 in the band vertical order.
The filter processing unit 212 performs, for example, 3 × 3 filter operation on the pixel data rotated at an arbitrary angle while accumulating using a buffer, and performs noise removal and the like. The filtered image data is output to the outside.

  The skew correction unit 211 and the filter processing unit 212 require a certain amount of delay buffer for referring to surrounding pixels. However, the delay buffer capacity depends on the height of the band by processing in the band vertical order. . That is, according to the method of the present embodiment, it is possible to realize an image processing apparatus that does not depend on the size of the original image.

  As described above, according to the present embodiment, the aspect ratio of the write unit to the memory is adaptively changed according to the rotation angle θ. Thereby, the efficiency of burst reading can be increased. In particular, since the storage unit is determined in consideration of the scanning order in both shading correction and skew correction, stable system performance independent of the skew angle that can change from page to page can be realized.

  Note that the pre-stage image correction unit 200 may have a function of discarding tiles unnecessary for the rotation process. For example, the tile 710 at the upper left end of the band 1 in FIG. Therefore, a configuration for discarding such unnecessary tiles may be provided separately.

  Further, the scanning order of the pixels is not limited to the order described in this embodiment. In general, when the scan order is output after the rotation process as a predetermined order that does not depend on the rotation angle, the input to the rotation process is an oblique input order that depends on the rotation angle. At this time, if the rotation angle θ is known when the rotation target image is written in the memory, the present invention can be similarly applied.

  Furthermore, the shading correction unit 202 and the filter processing unit 212 may be replaced with other processes, or other processes may be added. For example, optical distortion correction may be performed instead of shading correction, or resolution conversion may be performed after filter processing.

In this embodiment, the image data (tile data) is written in the memory 220 in a grid pattern, but the relative position may be shifted depending on the angle. FIG. 9 is a diagram showing an example of writing in which the positions are shifted by 2 × 8 tiles. FIG. 9A shows a case where each row is shifted in the main scanning direction, and FIG. 9B shows a case where each row is shifted in the sub scanning direction. Each case is shown. In this way, storing data while shifting it by a predetermined number of pixels may make the memory access more efficient, although the control such as the calculation of the memory address is slightly complicated.
Note that the efficiency in writing to the memory can be reduced by applying the present invention. However, rotation processing at an arbitrary angle generally refers to surrounding pixels, and there are often processes that refer to surrounding pixels such as filter processing and resolution conversion at subsequent stages. Therefore, it can be said that the memory read amount is usually larger than the memory write amount. That is, even if the memory writing efficiency is sacrificed, the total efficiency can be improved by improving the memory reading efficiency.

  Next, a mode in which shading correction is performed in a scanning order different from that in the first embodiment will be described as a second embodiment. The description of the parts common to the first embodiment will be omitted or simplified, and the differences will be mainly described below.

  FIG. 10 is a block diagram illustrating an internal configuration of the image processing apparatus according to the present embodiment, and corresponds to FIG. 2 in the first embodiment. The scanner correction processing of the present embodiment is roughly the same as that of the first embodiment in that the scanner correction processing is roughly divided into processing in the preceding image correction unit 1000 and processing in the subsequent image correction unit 1010. Compared to the configuration of FIG. 2 according to the first embodiment, it can be seen that the HV conversion unit is removed in the pre-stage image correction unit 1000 of the present embodiment.

  The input of image data to the pre-stage image processing unit 1000 in this embodiment is assumed to follow the tile horizontal order. FIG. 11 is a diagram for explaining the tile horizontal order in this embodiment, and shows that each divided tile is scanned in the main scanning direction.

  The shading correction unit 1002 performs shading correction on each pixel in the tile horizontal order. Since the shading parameters differ between the left and right pixels, it is necessary to update each pixel. Therefore, a shading parameter corresponding to the tile width is accumulated in a temporary memory or the like in the shading correction unit, and is read out and used for each pixel.

  In the reorder unit 1003, like the reorder unit 203 in the first embodiment, the pixels are packed into a memory storage unit suitable for the rotation angle, and are written in the memory 1020. Therefore, the reorder unit 1003 includes a temporary memory capable of storing a plurality of tile-width rows, and writes the data to the memory 1020 as needed as soon as the pixels of the memory storage unit are aligned in the temporary memory.

  The image at the time of writing in the memory 1020 is the same as that in the first embodiment, and the subsequent processing is not different from that in the first embodiment. That is, since the data is stored in the memory 1020 in a memory storage unit suitable for the rotation process, the memory can be read efficiently.

  As described above, the present invention can be similarly applied even in the case of a generally used horizontal scanning order.

(Other examples)
The present invention is also realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (12)

Means for performing first image processing on the input image in a predetermined scanning order;
A memory for storing the image subjected to the first image processing in a predetermined storage unit;
Means for performing second image processing including rotation processing at an arbitrary angle on an image read out in bursts in the predetermined storage unit from the memory;
Determining means for determining the predetermined storage unit according to the arbitrary angle;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the determination unit selects and determines the predetermined storage unit from a plurality of candidates having different aspect ratios. The plurality of candidates is a rectangular region of vertical P pixels and horizontal Q pixels, which is a combination of natural numbers P and Q satisfying N = PxQ, where the burst length when accessing the memory is N pixels. the image processing apparatus according to claim 2, wherein. The determination unit selects, as the predetermined storage unit, an angle formed by a diagonal line of the rectangular area and a scanning direction according to the predetermined scanning order that is closest to the arbitrary angle among the plurality of candidates. The image processing apparatus according to claim 3. The determination unit selects the predetermined storage unit corresponding to the arbitrary angle with reference to a table that defines a correspondence relationship between a plurality of different angles and the rectangular area. The image processing apparatus described. It said first image processing, the image processing apparatus according to any one of claims 1 to 5, characterized in that a shading correction. Wherein the predetermined scan order, each obtained by dividing the input image in the sub-scanning direction region, any one of claims 1 to 6, characterized in that the order in which repeated scanning in the sub-scanning direction in the main scanning direction The image processing apparatus according to item 1. The input image is an image obtained by scanning a document with a scanner,
Further comprising skew detection means for detecting an inclination of the document set on the scanner,
The image processing apparatus according to claim 1, wherein the arbitrary angle is an angle detected by the skew detection unit.   The image processing apparatus according to claim 1, wherein the predetermined scanning order is an order unique to the image processing apparatus.   The image according to any one of claims 1 to 9, wherein the storage unit stores the storage unit by shifting the storage unit by a predetermined pixel for each row or column according to the arbitrary angle. Processing equipment. Performing first image processing on the input image in a predetermined scanning order;
Storing the image subjected to the first image processing in a memory in a predetermined storage unit;
Performing second image processing including rotation processing at an arbitrary angle on an image read out in bursts from the memory in the predetermined storage unit;
A determining step of determining the predetermined storage unit according to the arbitrary angle;
An image processing method comprising:   A program for causing a computer to function as the image processing apparatus according to any one of claims 1 to 10.

Images