JP2010244354A - Image processing apparatus and processing method, and processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect an angle of inclination of an input image from properties of the input image itself, and to eliminate the influence of oblique lines etc., in the input image. <P>SOLUTION: An edge is extracted from the input image. The extracted edge is binarized, and degeneration processing is carried out to remove connected effective pixels. Through degeneration processing, for example, a segment is simplified up to two effective pixels. By having the image having been subjected to the degeneration processing Hough-transformed, a plurality of segments are extracted. The angle of inclination of the input image is detected from inclinations of the plurality of extracted segments. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to detection of an inclination angle of an input image.

  When an image is read by a scanner, the image may be read in a slightly inclined state. In order to correct this, the side of the original paper is detected. If the original paper is square, the direction of the side should be parallel to the main scanning direction or the sub-scanning direction, and the deviation from these directions represents the inclination of the original paper. However, the inclination can be detected by this method only at the time of scanning, and the inclination of the accumulated image and the image transferred from the outside cannot be detected. Further, since the inclination of the original paper is detected, the inclination cannot be detected when the paper is not a fixed shape such as a cutout. Furthermore, when scanning a book with a hard cover in a two-page spread, the cover may not be tilted, but the spread page may be tilted. In this case, it is difficult to detect the inclination. In order to solve these problems, it is necessary to detect the inclination of the image itself rather than detecting the inclination of the original paper.

  Here, showing related prior art, Patent Document 1: JPH10-283476A describes that a straight line in an image is detected by Hough transform. However, Patent Document 1 does not indicate that the inclination of the image itself is detected by the Hough transform. For example, if there are photographs or figures in the image and these include oblique lines, the oblique lines are extracted by the Hough transform, and an oblique line angle different from the inclination of the image itself is detected.

JPH10-283476A

  An object of the present invention is to obtain the tilt angle of an input image using the properties of the input image itself and without being affected by oblique lines or the like in the input image.

An image processing apparatus of the present invention is an apparatus for determining an inclination angle of an input image,
A reduction processing unit for applying a reduction process for simplifying a line segment to a point to an input image;
A Hough transform unit for performing a Hough transform on the image after the reduction process;
A tilt angle determining unit that determines the tilt angle of the input image from the direction of the line segment extracted by the Hough transform.

The image processing method of the present invention also determines the tilt angle of the input image.
Applying a reduction process to the input image to simplify line segments into points;
A step of Hough transforming the image after the reduction processing;
Determining the inclination angle of the input image from the direction of the line segment extracted by the Hough transform.

In addition, the image processing program of the present invention determines the tilt angle of the input image by the information processing device.
The information processing apparatus;
A reduction processing unit for applying a reduction process for simplifying a line segment to a point to an input image;
A Hough transform unit for performing a Hough transform on the image after the reduction process;
It is made to function as a tilt angle determination unit that determines the tilt angle of the input image from the direction of the line segment extracted by the Hough transform.

In this specification, the description relating to the image processing apparatus also applies to the image processing method and the image processing program as it is, and the description relating to the image processing method also applies to the image processing apparatus and the image processing program as they are.
In this specification, the point is not limited to one pixel in the image, but includes a point that can be seen as a point by a plurality of pixels.

  The line segment in the input image is converted into a simple point by the reduction process, but the character line is preserved. For this reason, the inclination angle can be determined by the line of characters in the input image without being affected by the oblique line or the like. Furthermore, since the number of effective pixels in the input image is reduced by the reduction process, the Hough transform can be executed at a higher speed.

Preferably, the Hough transform unit and the tilt angle determination unit perform a Hough transform on the input image subjected to the reduction process to determine a first tilt angle, and perform a Hough transform on the input image not subjected to the reduction process. To determine the second tilt angle and
A comparison unit between the first tilt angle and the second tilt angle is provided, and when the difference is within a predetermined range, the second tilt angle is set as the tilt angle of the input image, and when the difference is outside the predetermined range, the first Is the inclination angle of the input image.
Here, the first inclination angle represents the inclination angle obtained using only the character line, and the second inclination angle represents the inclination angle obtained using a table, underline, diagonal line, etc. in addition to the character line. Yes. Therefore, when the first inclination angle and the second inclination angle are approximated, it can be estimated that a line segment that is parallel or perpendicular to the character row in the table or underline contributes. Even if the inclination angle is obtained from the line of characters, the second inclination angle has not been subjected to the change of the input image due to the degeneration process, so it can be said that the accuracy is higher than the first inclination angle. Therefore, the tilt angle can be determined with high accuracy.
When the first tilt angle and the second tilt angle are significantly different, if the first tilt angle is the tilt angle of the input image, the first tilt angle is not affected by the oblique lines in the graphic portion or the photograph portion in the input image. The inclination angle can be determined.

  In the case where the consistency between the first tilt angle and the second tilt angle is intermediate between the above, for example, a plurality of line segments are taken out as a first set from the Hough transform of the image subjected to the degeneration process, and the degeneration process is performed. A plurality of line segments are taken out as a second set from the Hough transformed image that has not been subjected to. From the second set, a line segment whose direction difference is within a predetermined range is in the first set, and more preferably, a direction difference is within the first predetermined range and a distance difference in the Hough transform is A line segment within a predetermined range of 2 is extracted from the first set, and an inclination angle is determined. Although such a line segment may be one, it is more reliable to determine the tilt angle using a plurality of line segments. Therefore, when the number of extracted line segments is small, in order to increase reliability, preferably, for example, in addition to the extracted line segments that do not correspond to the second set in the first set, The inclination angle of the input image is determined from the inclination of the line segment.

Therefore, preferably, when the difference is outside the predetermined range,
A line segment whose slope is approximated between a first line segment group extracted by Hough transform from the input image subjected to the reduction process and a second line segment group extracted by Hough transform from the input image not subjected to the reduction process If there is no, the first tilt angle is the tilt angle of the input image,
If there is a line segment with an approximate slope between the first line segment group and the second line segment group, at least the line segment of the second line segment group and the slope approximate to the first line segment group Based on the inclination of a certain line segment, the inclination angle of the input image is determined.

Alternatively, without obtaining the first tilt angle and the second tilt angle,
The Hough transform unit extracts the first line segment group from the input image that has been subjected to the reduction process, and extracts the second line segment group from the input image that has not been subjected to the reduction process,
The inclination angle detection unit may determine an inclination angle of the input image based on an inclination of a line segment that has at least a line segment of the second line segment group and whose inclination approximates to the first line segment group. .

  More preferably, the inclination angle detection unit is a line segment of the second line segment group, and when there are more than a predetermined number of line segments having an inclination similar to the first line segment group, the line of the predetermined number or more. The inclination angle of the input image is determined based on the inclination of the minute, and when the number of line segments in the second line segment group that approximates the inclination to the first line segment group is less than the predetermined number, the predetermined value The inclination angle of the input image is determined based on the inclination of the line segment group obtained by adding the line segment of the first line segment group to the line segments less than the number.

Preferably, a binarization unit that binarizes the input image into effective pixels and invalid pixels is provided, and the degeneracy processing unit includes the connected effective pixels in the binarized input image from the end. A degeneration process is executed by a filter for replacing with invalid pixels.
In this way, the reduction process can be easily performed.

Block diagram of the MFP of the embodiment Block diagram of the pre-processing unit in the embodiment The figure which shows the filter used by the degeneracy process of an Example The figure which shows the algorithm of determining an inclination angle from a table | surface and an underline in an Example, or determining an inclination angle using a character. The figure which shows the pre-processing algorithm in an Example Block diagram of Hough transform unit to image rotation unit in the embodiment The figure which shows typically the voting table in an Example. The figure which shows the algorithm of the inclination-angle detection in an Example The figure which shows that the line of the character is regarded as a straight line in the embodiment The figure which shows the example which binarized the input image directly The figure which shows the reduced image of the input image in an Example The figure which shows the image which removed the noise from the reduction image in an Example. The figure which shows the edge extracted from the image after a noise removal in an Example. The figure which shows the edge extraction filter which was used in the execution example The figure which shows the image which binarized the image of an edge in an Example The figure which shows the image which removed the isolated point from the binarized image in an Example. The figure which shows the image which performed the reduction process after removing an isolated point in an Example. Illustration showing fake maximum voting line The figure which shows the condition where two or more neighborhood points in a voting table are selected as an effective line In an Example, it is a figure which shows the extraction condition of an effective line when masking the vicinity of the extracted effective line, and the lightness shows the number of votes in a voting table The figure which shows typically the extracted effective line in an Example The figure which shows typically the determination method of the inclination-angle in an Example

  In the following, an optimum embodiment for carrying out the present invention will be shown.

  1 to 22 show an embodiment of the multifunction machine 2 as an example. 4 is a LAN, 6 is a personal computer, which is an example of an information processing apparatus, and 8 is a router, which connects the Internet 10 to the multifunction device 2 and the personal computer 6. The detection and correction of the tilt angle in the embodiment is performed by the multi-function device 2, but may be performed by the personal computer 6 or the like. For example, an image read by the scanner 20 of the multi-function device 2 is transferred to the personal computer 6 and personal The computer 6 may detect and correct the tilt angle.

  The structure of the multifunction device 2 will be described. 12 is a LAN interface, 14 is a bus, 16 is an image memory, and stores image data as a page memory. The memory 18 is a general-purpose memory and stores programs, intermediate data, and the like. The scanner 20 reads an image from original paper in monochrome gray scale or full color, the printer 22 prints the image, and the G3 facsimile unit 24 performs G3 facsimile transmission and reception of the image. The e-mail unit 26 transmits and receives an image as an attached file of the e-mail, and the network server unit 28 operates the multifunction device 2 as a document management server or a remote printer. The user interface 30 accepts user manual input and the like, and the image processing unit 32 performs various image processing. In order to detect the tilt angle of the scanned image, the received image, or the accumulated image, a pre-processing unit 40, a Hough transform unit 50, and a tilt angle detection unit 60 are provided. Then, the image is rotated by the image rotation unit 70 by the same angle opposite to the detected inclination angle, and the inclination is corrected.

FIG. 2 shows the configuration of the preprocessing unit 40. Image data is input from the image memory 16 to the reduction processing unit 41, and the resolution of the image is reduced to 1 / n (n is a natural number of 2 or more, preferably 4 to 8, particularly 4 to 6). As a result, the amount of image data is reduced to 1 / n ^2. For reduction, for example, an average value of pixel values may be obtained for each block of n × n pixels. Further, more simply, the reduction processing unit 41 may output the data of one pixel of the representative point in the block of n × n pixels, for example, the upper left vertex of the block as it is. For this reason, the Hough transform is n ² times faster. Even if the content and resolution of the data are the same for the same image data, if the reduction rate in the reduction processing unit 41 is different, the detection result of the tilt angle may be different. Therefore, regardless of the resolution at the time of scanning, if the resolution is reduced to a certain resolution by the reduction processing unit 41, the difference in the detected value of the tilt angle due to the difference in resolution can be eliminated.

  If the resolution at the time of scanning is known, such as received data of a facsimile, other than the image scanned by the multifunction device 2, the image is similarly reduced to a constant resolution. The scanning resolution is often determined from the communication protocol or the header of the image file. When the resolution is not described in the header like a jpeg image, the image is reduced at a constant reduction rate such as 1/4 or 1/8.

  The smoothing unit 42 removes noise from the reduced image using a smoothing filter or the like, and does not extract random data in the image as a straight line by Hough transform. Next, the edge extraction unit 43 extracts edges from the smoothed image using an edge extraction filter such as a Laplacian filter or a Sobel filter. As a result, the influence of the ground color is eliminated, and an edge corresponding to the outline of the character can be extracted even with a white character.

  The binarization unit 44 binarizes the extracted edge. The binarization threshold value may be constant, or may be determined from the brightness at the extracted edge or the distribution of color values. Furthermore, it is not necessary to binarize all edges. For example, when a character part, an image part, and a graphic part in an image can be discriminated, it is preferable to binarize only the character part.

  The image data to be handled is a monochrome gray scale image in the embodiment, but may be an RGB image or the like. In this case, the RGB data may be converted into a brightness image to detect the inclination angle, or the inclination angle may be detected for each of R, G, and B components, and the average value thereof may be used.

  The isolated point removing unit 45 may remove isolated points from the binarized image and remove isolated points from the edge image before binarization. Removal of isolated points reduces the amount of data and reduces the probability of detecting false tilt angles. It should be noted that isolated points are particularly many in a photographic image and are data unrelated to the original inclination of the image.

  The reduction processing unit 46 performs a reduction process on the binarized image after the isolated points are removed. However, the removal of isolated points may be omitted. Here, the degeneracy processing means that effective pixels are continuously arranged in the vertical, horizontal, diagonal directions, that is, when effective pixels are connected vertically / horizontally / diagonally, the effective pixels are connected to the connected portion. It is a process of deleting from the end. In the reduction process, when effective pixels are connected to form a line segment, the effective pixel is deleted from the end of the line segment, and the line segment is simplified to a point composed of one pixel or two pixels. The effective pixel means a pixel with data in the binarized image, and it does not matter whether it is white or black. The line segment is simplified to, for example, two pixels by the reduction process. A circle, a square, a cross, and the like are also simplified to, for example, 2 pixels or 4 pixels if effective pixels are connected to each other. As a result, it is possible to prevent the oblique lines in the graphic part and the photographic part in the image from disappearing and to affect the detection of the inclination angle. In the reduction process, the number of effective pixels of the character is reduced, but the straight line formed by the character line is not lost. Further, the reduction process reduces the number of pixels to be subjected to the Hough transform, thereby speeding up the process.

  FIG. 3 shows filters 34 to 38 used for the degeneration processing. The image to be processed is already binarized, and is a target pixel for determining whether or not a hexagonal pixel is replaced from an effective pixel to an invalid pixel (invalid pixel). When the pixel marked with ○ is an effective pixel and one of the pixels marked with Δ is a valid pixel, the target pixel is replaced with an invalid pixel. In the reduction process, when one line is processed along the main scanning direction, the next line is processed by moving one pixel in the sub-scanning direction. In the degeneration processing, for example, the five filters 34 to 38 are processed in parallel, and the target pixel corresponding to one of the filter patterns is replaced with an invalid pixel. However, the order and direction of processing are arbitrary.

  The filters 34 to 38 make one pixel at the end of three effective pixels connected vertically / horizontally / diagonally as invalid pixels. In general, a filter used for the degeneration processing is one in which at least one pixel at the end is an invalid pixel from a plurality of effective pixels connected vertically, horizontally, or diagonally. In the degeneration process, one pixel or a plurality of pixels from the four or more effective pixels connected in an arbitrary direction, not limited to the vertical and horizontal directions, even if one pixel is selected from two connected effective pixels as an invalid pixel. May be invalid pixels.

  Although it is easy to apply a reduction process to a binarized image, for example, an image before binarization such as after smoothing or after edge extraction can be applied to the reduction process. For example, let us consider that the lightness p is a threshold for binarization and a pixel having lightness of p or less is an effective pixel. In the filters 34 to 38, if the pixel at the position of the target pixel is lightness p or less, it is determined as the target pixel, and if the pixels at the positions ◯ and Δ in the figure are lightness p or less, they are valid pixels. Then, degeneracy and binarization can be processed collectively. The reduction process has an effect similar to edge extraction in that the number of pixels is reduced, and may be applied to the image after smoothing. In this case, edge extraction can be omitted. Further, the reduction process may be performed on an image after reduction and before smoothing. In an extreme case, the image before reduction may be subjected to reduction processing, binarized, and Hough converted. In this case, reduction, smoothing, edge extraction, and removal of isolated points are omitted.

  By the reduction process, the inclination angle A obtained from the straight line in the image can be compared with the inclination angle B obtained from the line of characters. When the angles A and B substantially coincide with each other, the angle A represents the direction of a ruled line, the direction of an underline, or the direction of a character line parallel to or perpendicular to a character line in the image. On the other hand, the angle B represents only the direction of the line of characters in the image. When the angle A and the angle B substantially coincide with each other, for example, if the angle A indicates a front or underline direction, the angle A is more meaningful than the angle B. An algorithm for comparing the angles A and B and determining the tilt angle is shown in FIG.

  Step 1 binarizes the image, Step 2 Hough transforms, Step 3 extracts a plurality of line segments in the image (for example, 8 or 16 lines) to form a line set A, Step 4 The inclination angle A is determined. On the other hand, after the binarized image is subjected to reduction processing in step 5, Hough transform is performed in step 6, and a plurality of line segments in the image are extracted (for example, 8 or 16 lines) in step 7 to set line segment B In step 8, the inclination angle B of the image is determined. In step 9, it is determined whether or not the angles A and B are substantially equal, for example, whether or not the difference is within ± 1 °, more broadly within ± 3 °. If the angles A and B are approximately equal, An inclination angle is set (step 10). If the angles A and B are not substantially equal, it is determined in steps 11 and 12 whether there are more than a predetermined number of line segments, for example, three or more, that have not been subjected to the degeneration process and those that have been subjected to the reduction process. . The approximate line segment is, for example, a line segment in which the difference in θ is within 1 ° and the difference in ρ is within a predetermined value, for example, within 10 pixels.

  If there is a line segment that approximates the predetermined number or more, the inclination angle is determined from the angle of the approximate line segment on the side that has not been subjected to the degeneration process (step 13). In step 13, preferably, the line segment extracted from the side not subjected to the reduction process is used, and the angle is determined by adding the number of votes for each line segment to the weight or the like. If there is an approximate line segment but less than the predetermined number, the inclination angle is determined from the angle of the approximate line segment on the side not subjected to the reduction process and the other line segment on the side subjected to the reduction process (step 14). However, if there is a line segment on the side that has been subjected to the reduction process and the line segment to be approximated is on the side that has not been subjected to the reduction process, it should not be used to avoid using similar data in layers. Is preferred. In step 14, for example, the product of the parameter determined based on whether or not the line segment has not been subjected to the reduction process and the number of votes for each line segment is used as a weight for each line segment, and the side on which the reduction process has not been applied. For example, a weight of 2 to 4 times is added to the line segment as a parameter. If there is no approximate line segment, the angle B is output on the side where the reduction process has not been performed, assuming that the graphic portion or the oblique line in the photograph has been extracted (step 15). The determination algorithm for the angles A and B itself is shown in FIG. 8. When the line segment used for determining the angle is determined in steps 13 and 14, the algorithm in FIG. 8 is executed again. Here, the line segment in FIG. 4 corresponds to a point in the (ρ, θ) space in FIG.

  In the embodiment, the reliability is increased by determining the tilt angle from a plurality of line segments in accordance with the algorithm of FIG. 8, but the tilt angle may be determined from a single line segment. In this case, if step 11 is yes, the inclination angle is determined from, for example, one line segment on the angle A side (side on which the degeneration process has not been performed), and steps 12 and 14 are unnecessary.

  Without obtaining the angles A and B, step 11 can be executed after step 3 and step 7 of FIG. In this case, steps 4, 8, 9, 10, and 15 are omitted in FIG. 4, and when there is no line segment that approximates A and B in step 11, step 8 is executed for the first time, and the angle B is output.

FIG. 5 shows a control algorithm of the preprocessing unit 40. In step 21, the image is read by the scanner 20, and in step 22, the image is reduced to a certain resolution, thereby improving the Hough transform processing speed by n ² times. In step 23, noise is reduced by smoothing, and in step 24, an edge is extracted to remove the influence of the ground color, etc., and white characters and the like can be detected. Then, binarization is performed in step 25, and isolated points are removed in step 26. This can reduce the influence of an isolated point such as a photographic image on the detection of the tilt angle. The removal of isolated points may be performed before binarization. In step 27, the data is simplified by reduction processing, and in particular, the straight line of the photograph part and the graphic part is simplified to, for example, two pixels. Step 27 may be executed before any of steps 22 to 26.

  The pre-processing of Step 22 to Step 27 is performed in synchronization with image scanning, and the Hough conversion by the Hough conversion unit 50, the detection of the inclination angle by the inclination angle detection unit 60, and the inclination angle by the image rotation unit 70 are performed. Corrections can be made in synchronization with the scan. Here, “synchronization” means that these processing speeds are equal to or higher than the reading speed of the scanner 20, and in this way, the inclination of the image can be corrected in real time simultaneously with the scanning of the image.

  FIG. 6 shows the structure of the Hough transform unit 50 to the image rotation unit 70. The Hough transform unit 50 includes, for example, 16 to 32 execution units 51, and executes the Hough transform in parallel. Also, the Hough transform is performed on both the image after the reduction process and the image before the process. The execution unit 51 includes a ρ value calculation unit 52 and a table 53 for sin and cos. The angle range stored in one table 53 is, for example, 5.625 ° (180/32) so that the 32 tables 53 can cover 0 ° to 180 °, and the angle increment is, for example, 0.176. ° (180 ° / 1024). The ρ value calculation unit 52 reads the values of sin θ and cos θ from the table 53 and sets the ρ value as ρ = x cos θ + y sin θ for the effective pixel position (x, y) output from the degeneration processing unit 46 and the isolated point removal unit 45. calculate. The voting table 54 is a two-dimensional table of θ and ρ, and the management unit 55 manages a plurality of execution units 51.

  FIG. 7 shows the relationship between the ρ value calculation unit 52 and the voting table 54. Each point in the table 54 represents a combination of ρ and θ, which represents a line segment. Assume that ρ and θ are each divided into 1024 levels, and when x and y values are input, the value of ρ is calculated for each θ and 1 is added to the data at the corresponding position in the table. In addition, θ decomposes the range of 180 ° into, for example, 256 to 4096 level, preferably 1024 to 2048 level, and the inclination angle is about 0.05 ° to 0.3 °, preferably about 0.1 ° to 0.2 °. Make detection with accuracy. Further, the resolution of ρ may be lower than the resolution of θ. The execution unit 51 has, for example, 32 units, and each ρ value calculation unit 52 takes charge of θ for 32 levels (1024/32). As a result, for example, the (ρ, θ) value is voted as indicated by a circle in the third column from the left. Reference numeral 57 denotes a mask, which masks the periphery of a point already extracted as an effective line (extracted straight line) from the table 54 so that the next effective line is extracted from the outside of the mask 57.

  Returning to FIG. 6, the effective line extraction unit 61 extracts, for example, the upper m-th point from the voting table 54. A point with a large number of votes in the voting table 54 is likely to be an effective line in the image. Then, a mask 57 is applied around the extracted points so that other points are not extracted from the mask 57. The candidate angle extraction unit 62 extracts inclination angle candidates (k, for example, m / 2 or more) from the extracted effective lines, and the determination unit 63 determines the inclination angle θd based on these candidate angles. The determined inclination angle θd is input to the coordinate calculation unit 71, and the coordinates (address) after inclination correction are calculated for each pixel of the image stored in the image memory 16. The address generation unit 72 generates a read address to the image memory 16 based on the coordinates before tilt correction. The bicubic processing unit 73 interpolates the read data, and writes the interpolated data back to the image memory 16 based on the coordinates after the inclination correction. By these processes, the image in the image memory 16 is rotated so as to correct the inclination. A simple affine transformation or the like may be used for the rotation.

  FIG. 8 shows an algorithm in the tilt angle detection unit 60, and points from the voting table consisting of the upper m-th place, for example, the 16th or eighth place (ρ, θ) are extracted (step 31). Then, the vicinity of the extracted points, for example, the vicinity of a square having 5 to 40 levels on one side is masked (step 32) so that two or more points are not extracted from the same vicinity. It should be noted that only (ρ, θ) having a vote count of ½ or more of the maximum vote count is extracted. Therefore, if the number of votes at the m-th point is less than ½ of the maximum number of votes, an error is detected, the inclination angle is not detected, and the image rotation unit 70 does not rotate the image (step 33). Steps 32 and 33 are auxiliary processes for executing step 31. In this way, m points that are not in the vicinity of other points and that are 1/2 or more of the maximum number of votes are extracted. 1/2 or more is an example. For example, it may be 2/3 or more or 0.4 or more. Steps 31 to 33 correspond to the processing of the effective line extraction unit 61.

  Next, an angle for obtaining a tilt angle candidate is θx, and its orthogonal angle is φx. Here, φx is positive or 0, and φx = θx + π / 2 or φx = θx−π / 2. For example, δ is 1 ° or 0.5 °, and θx is determined so that the number of points (ρ, θ) within the range of θx ± δ and φx ± δ is maximized. . Then, the points that fall within the ranges of θx ± δ and φx ± δ are output (step 34). In other words, the angle at which the number of points having the same angle θ component in the range of ± δ is the maximum for m points is defined as θx. Instead of simply considering the number of points where the θ components coincide within the range of ± δ, the sum of the votes may be maximized for the points falling within the range of θx ± δ and φx ± δ. . If the number k of these points is less than the threshold, for example, less than m / 2 or less than m / 3, an error is determined in step 35. In the case of an error, the inclination angle is not detected, and the image rotation unit 70 does not rotate the image. Steps 34 and 35 correspond to the processing of the candidate angle extraction unit 62.

  Step 36 corresponds to the processing of the determination unit 63 and outputs, for example, a simple average value of θ as the inclination angle θd for the k points (ρ, θ) obtained in step 34. Alternatively, a weighted average of θ at k points may be used, and the number of votes vi may be used as the weight. Instead of these, among the k θ candidates θ1 to θk, θ having the maximum number of votes vi may be used as the inclination angle θd. In step 36, θd is output. If θd exceeds π / 2, θd−π / 2, that is, φd is output.

  9 to 22 show the operation of the embodiment. The upper part of FIG. 9 shows the original image, and the character string in the image is regarded as a straight line, and its inclination is detected. Therefore, the object of the embodiment is to convert the upper image in FIG. 9 into the lower image in FIG. 9 and perform Hough transform on the image.

FIG. 10 shows an example in which an input image (an image in the image memory 16) is directly binarized without performing reduction, smoothing, or edge extraction. As marked with a circle, black pixels such as a photo portion other than characters are shown. The number of (effective pixels in this case) increases, the burden of Hough conversion increases, and the influence of oblique lines in the photograph becomes stronger, resulting in a decrease in detection accuracy. Therefore, the input image is handled in gray scale and subjected to image processing such as reduction, smoothing, and edge extraction, and then binarized. In order to perform the Hough transform at high speed, the input image is reduced to 1 / n, thereby reducing the subsequent calculation time to 1 / n ² (FIG. 11).

  Next, noise is removed by smoothing (FIG. 12), and an edge is extracted to obtain the image of FIG. By extracting the edge, the influence of the ground color is eliminated, and it is possible to cope with white characters, and further simplifies the data and facilitates the Hough conversion.

  FIG. 14 shows an edge extraction filter 80. By applying the filter 80 from the upper left side to the processed image 81, the lower and right edges 82 of the processed image 81 are extracted. As a result, the amount of edges can be halved compared to the case of detecting the entire edge of the image 81 to be processed. As described above, preferably, two types of edges are extracted from among the four types of edges, top, bottom, left, right, bottom and left, top and right, and top and left. When the image after edge extraction is binarized, the image of FIG. 15 is obtained. When the isolated points are removed from the image of FIG. 15, the image of FIG. 16 is obtained. The isolated points may be removed from the edge image shown in FIG.

  As shown in FIG. 15 and FIG. 16, if the graphic part line or the photographic part line remains, it may cause an erroneous detection of the inclination angle. Therefore, when the reduction process is performed, the image of FIG. 17 is obtained, lines in the photograph and the figure are converted into isolated points, and a line made up of a string of characters is saved. Therefore, by performing Hough transform on the image after the reduction processing, only a straight line composed of character lines can be detected. However, if there is a table or underline in which the character lines and directions are aligned as in step 10 of FIG. 4, it is preferable to detect the tilt angle using the table or underline. For this reason, in terms of mounting, it is preferable to perform Hough transform on both the image after the reduction process and the image before the reduction process.

  If data is accumulated in the voting table by the Hough transform and only the point with the largest number of votes is simply extracted from the voting table, erroneous detection may be performed. FIG. 18 shows such an example. In FIG. 18, since only a part of the image is displayed, it is difficult to know where the “maximum voting line” is extracted, but it is extracted from a photograph part or the like in the image. is there. For this reason, a plurality of points with the highest number of votes are extracted. If points up to the top m are simply extracted here, a plurality of neighboring points in the voting table are often extracted. Therefore, the highest point is extracted from the voting table 54, the neighborhood of the extracted point is masked, the point of the next vote number is extracted, and the neighboring point is extracted again. Extract. If the lowest point of the extracted top m ranks does not have, for example, ½ or more votes of the highest point, the detection is regarded as an error. When a line consisting of characters is extracted, there should be no significant difference in the number of votes between the maximum number of votes and the m-th place from the top. Despite this, there is a great difference that the highest point is likely to be a straight line portion in the figure. Alternatively, from the top m ranks (for example, 16th rank), a point having a predetermined number of votes (for example, 1/2 or more) with respect to the highest point is extracted, and the number of extracted points is predetermined. When the value is equal to or less than k (for example, 4), the tilt angle may not be detected as an error.

  FIG. 20 schematically shows the situation of the voting table. Here, the number of votes is represented by brightness. The vicinity of the selected point is covered with a black square mask. In FIG. 20, the top 8 points have already been extracted.

  An inclination angle is determined for the extracted upper m-th point. This procedure is schematically shown in FIGS. 21 and 22, where m is 8, for example. When a character row is detected, the upper and lower columns of the character block can be detected at the same time, so that the inclination angle θd and its orthogonal angle φd are significant. In addition, the tolerance for extracting candidates is δ (about 0.5 ° or 1 °), and θx is determined so that the number k of points within the range of angles θx ± δ and φx ± δ is maximized. To do. Here, if the value of k is less than a threshold value (for example, m / 2) for any angle, an error is assumed, for example. When an angle θx including a point equal to or greater than the threshold is obtained, the candidate angle is determined by a simple addition average or a weighted average of the angles of the included points. This is the process of step 36 in FIG.

  When the tilt angle θd is determined with an accuracy of, for example, about ± 0.1 ° as described above, the image is rotated by the image rotation unit 70 to correct the tilt. The image rotation itself is a simple process and can be executed in synchronization with, for example, scanning.

In the embodiment, the example in which the rotation and the correction of the tilt angle are performed inside the multi function device 2 has been described. However, it may be performed by the personal computer 6 or the like. In that case, when the image processing program of the embodiment is executed by an information processing apparatus such as a personal computer, the information processing apparatus becomes the image processing apparatus of the present invention.
When detecting the inclination of a book spread, for example, if it is input that the type of the document is a page spread before scanning, separate left and right inclination angles can be detected. Alternatively, the input image may be divided into right and left to obtain an inclination angle, and if the left and right inclination angles are different, it may be estimated that the image is a spread image.

2 MFP 4 LAN
6 personal computer 8 router 10 internet 12 LAN interface 14 bus 16 image memory 18 memory 20 scanner 22 printer 24 G3 facsimile unit 26 e-mail unit 28 network server unit 30 user interface 32 image processing unit 34 to 38 filter 40 preprocessing unit 41 Reduction processing unit 42 Smoothing unit 43 Edge extraction unit 44 Binarization unit 45 Isolated point removal unit 46 Reduction processing unit 50 Hough transform unit 51 Execution unit 52 ρ value calculation unit 53 Table 54 Voting table 55 Management unit 56 Data 57 Mask 60 Inclination Corner detection unit 61 Effective line extraction unit 62 Candidate corner extraction unit 63 Determination unit
70 image rotation unit 71 coordinate calculation unit 72 address generation unit 73 bicubic processing unit 80 filter 81 processed image 82 edge

Claims (8)

An apparatus for determining an inclination angle of an input image,
A reduction processing unit for applying a reduction process for simplifying a line segment to a point to an input image;
A Hough transform unit for performing a Hough transform on the image after the reduction process;
An image processing apparatus comprising: an inclination angle determination unit that determines an inclination angle of an input image from the direction of a line segment extracted by Hough transform. The Hough transform unit and the tilt angle determination unit perform a Hough transform on the input image subjected to the reduction process to determine a first tilt angle, and perform a Hough transform on the input image that has not been subjected to the shrink process. Configured to determine the tilt angle of the
A comparison unit between the first tilt angle and the second tilt angle is provided, and when the difference is within a predetermined range, the second tilt angle is set as the tilt angle of the input image, and when the difference is outside the predetermined range, the first The image processing apparatus according to claim 1, wherein an inclination angle of the input image is set to be an inclination angle of the input image. If the difference is outside the predetermined range,
A line segment whose slope is approximated between a first line segment group extracted by Hough transform from the input image subjected to the reduction process and a second line segment group extracted by Hough transform from the input image not subjected to the reduction process If there is no, the first tilt angle is the tilt angle of the input image,
If there is a line segment with an approximate slope between the first line segment group and the second line segment group, at least the line segment of the second line segment group and the slope approximate to the first line segment group Based on the inclination of a certain line segment, the inclination angle of the input image is determined.
The image processing apparatus according to claim 2, wherein the tilt angle detection unit is configured. The Hough transform unit extracts the first line segment group from the input image that has been subjected to the reduction process, and extracts the second line segment group from the input image that has not been subjected to the reduction process,
The inclination angle detection unit determines an inclination angle of an input image based on an inclination of a line segment having at least a line segment of the second line segment group and an inclination similar to that of the first line segment group. The image processing apparatus according to claim 1. The inclination angle detection unit determines the inclination of the line segment greater than the predetermined number when the line segment of the second line segment group has a line segment whose inclination approximates the first line segment group. And determining the inclination angle of the input image based on the input image, and if the line segment of the second line segment group has an inclination similar to that of the first line segment group and the line segment is less than the predetermined number, 5. The image processing apparatus according to claim 4, wherein an inclination angle of the input image is determined based on an inclination of a line segment group obtained by adding a line segment of the first line segment group to the minute. A binarization unit that binarizes the input image into valid pixels and invalid pixels is provided, and the degeneration processing unit replaces the valid pixels connected to each other in the binarized input image with invalid pixels from the end. An image processing apparatus according to claim 1, further comprising a filter for performing the processing. A method for determining an inclination angle of an input image,
Applying a reduction process to the input image to simplify line segments into points;
A step of Hough transforming the image after the reduction processing;
And determining an inclination angle of the input image from the direction of the line segment extracted by the Hough transform. In order to determine the tilt angle of the input image by the information processing device,
The information processing apparatus;
A reduction processing unit for applying a reduction process for simplifying a line segment to a point to an input image;
A Hough transform unit for performing a Hough transform on the image after the reduction process;
An image processing program that functions as an inclination angle determination unit that determines an inclination angle of an input image from the direction of a line segment extracted by Hough transform.