JP5477541B2 - Image processing apparatus, processing method, and processing program - Google Patents

Description

  The present invention relates to detection of an inclination angle of an input image.

  When an image is read by a scanner, the image may be read in a slightly inclined state. In order to correct this, the side of the original paper is detected. If the original paper is square, the direction of the side should be parallel to the main scanning direction or the sub-scanning direction, and the deviation from these directions represents the inclination of the original paper. Since the tilt can be detected by this method only at the time of scanning, the tilt of the stored image or the image transferred from the outside cannot be detected. Further, since the inclination of the original paper is detected, the inclination cannot be detected if the paper is not a fixed shape such as a cutout. Furthermore, when scanning a book with a hard cover in a two-page spread, the cover may not be tilted, but the spread page may be tilted. Even in this case, it is difficult to detect the inclination. In order to solve these problems, it is necessary to detect the inclination of the image itself rather than detecting the inclination of the original paper.

  Here, showing related prior art, Patent Document 1: JPH10-283476A describes that a straight line in an image is detected by Hough transform. However, Patent Document 1 does not indicate that the inclination of the image itself is detected by the Hough transform. Further, according to the experiment by the inventors, if the straight line with the maximum number of votes is a straight line parallel to the inclination of the image in the voting table obtained by the Hough transform of the image, the actual inclination of the image is often significantly different.

JPH10-283476A

  An object of the present invention is to accurately detect the tilt angle of an image using the properties of the image itself.

The image processing apparatus according to the present invention extracts a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as headings. In an image processing apparatus provided with a Hough conversion unit,
Extracting a predetermined number of points with the highest number of votes from the vote table;
The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx ,
Furthermore, when extracting the predetermined number of points, the neighborhood of the extracted points in the voting table is masked, and other points within the masked range are not extracted from the voting table.

The present invention relates to an image processing method for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as headings. In
Extracting a predetermined number of points with the highest number of votes from the voting table, masking the neighborhood of the extracted points in the voting table so that other points in the masked range are not extracted from the voting table,
Further , the extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx.

In the image processing program of the present invention, the image processing program is read into the information processing apparatus,
A function for converting the coordinates of a point in the image into a distance ρ and an angle θ, and voting on a voting table having the distance ρ and the angle θ as headlines;
A function of extracting a predetermined number of points with the highest number of votes from the voting table, masking the neighborhood of the extracted points in the voting table, and preventing other points within the masked range from being extracted from the voting table When,
Further, the function of determining the angle θx for extraction so that the consistency with the angle θ of the predetermined number of points is maximized, and obtaining the tilt angle of the image based on the determined angle θx for extraction is executed. Let

In this specification, the description regarding the image processing apparatus also applies to the image processing method and the image processing program as it is, and the description regarding the image processing method also applies to the image processing apparatus and the image processing program. The processing target image is, for example, an image immediately after being read by a scanner, but may be an image received by facsimile, file transfer, e-mail, or the like, or an image already stored in a memory. A straight line and a line segment have the same meaning.
The maximum consistency is, for example, that 1 is assigned to each point if it is within the allowable range from the extraction angle θx, and 0 if not, and the sum of these values is maximized. Instead of setting the value assigned here to 0/1, the value may gradually change due to an error from the extraction angle θx. Further, the number of votes obtained at each point may be used as the consistency weight, and the total number of votes obtained within the allowable range from the extraction angle θx may be regarded as consistency.

In the present invention, the inclination angle is determined from the direction of the line of the character, the ruled line, the underline, and the like based on the fact that the line formed by the character in the image can be regarded as an unbroken line in the Hough transform. Therefore, the tilt angle can be determined based on the nature of the image itself.
In the Hough transform, the coordinates (x, y) of the points in the image are converted into a distance ρ and an angle θ, with ρ = x cos θ + y sin θ. One point in the image is converted into a plurality of points in the (ρ, θ) space, and each converted point is voted on a voting table having ρ and θ as headings. Then, ρ and θ at points with a large number of votes represent straight lines in the image. However, the inventor has experienced that when the inclination of an image is determined from the point with the largest number of votes, an inclination angle with a direction significantly different from the direction of the row is often detected. Many of the detected tilt angles are due to straight lines of graphic parts or photographic parts included in the image, handwritten diagonal lines, and the like. On the other hand, if the extraction angle θx is determined so that the consistency with the angle θ of the upper predetermined number of points is maximized, and the inclination angle is determined based on the angle θx, the conspicuousness in the image is obtained. Since the inclination angle can be determined from a plurality of straight lines, more reliable detection was possible.
When extracting a predetermined number of points, the neighborhood of the extracted points in the voting table is masked so that other points in the masked range are not extracted from the voting table. Since straight lines can be extracted, the chances of extracting straight lines composed of character lines, ruled lines, underlines, etc. are increased.

Preferably, the inclination angle of the image is determined by averaging the angles θ with respect to the points within the predetermined number of points that include the angle θ within the allowable range from the extraction angle θx.
In this way, since the inclination angle can be determined by the average of the data of points within the allowable range, reliable detection can be performed.
Further, preferably, the average of the image is reflected by an average reflecting a weight corresponding to the number of votes of the angle θ with respect to a point that includes the angle θ within an allowable range from the angle θx for extraction among the predetermined number of points. Determine the angle of inclination. The average reflecting the weight according to the number of votes is, for example, a weighted average based on the number of votes, a weighted average based on the root or the square of the number of votes, and the like.
In this way, it is possible to determine the tilt angle while appropriately emphasizing points with a large number of votes.

Furthermore, the image processing apparatus according to the present invention converts the coordinates of the points in the image into a distance ρ and an angle θ, and votes on a voting table having the distance ρ and the angle θ as headlines, thereby obtaining a straight line in the image. In an image processing apparatus including a Hough conversion unit for extraction,
Extracting a predetermined number of points with the highest number of votes from the vote table;
The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx,
Furthermore, when the ratio between the number of points within the allowable range from the extraction angle θx within the predetermined number of points and the predetermined number is less than a predetermined value, the inclination angle of the image is detected. It is characterized by not being configured.
The present invention also resides in an image processing method corresponding to the image processing apparatus and an image processing program corresponding to the image processing apparatus.
The fact that only a small number of points are included within the allowable angle from the extraction angle θx means that the angle distribution of the extracted points is wide and the reliability is low. Therefore, in such a case, detection with low reliability can be avoided by terminating the detection of the tilt angle.

  The image processing apparatus according to the present invention extracts a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as headings. In an image processing apparatus provided with a Hough conversion unit,
  Extracting a predetermined number of points with the highest number of votes from the vote table;
  The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx.
  Further, when the predetermined number of points includes a point where the ratio of the number of votes with the highest number of votes is less than an allowable value, the inclination angle of the image is not detected.
The present invention also resides in an image processing method corresponding to the image processing apparatus and an image processing program corresponding to the image processing apparatus.
  When a line of characters or a ruled line in an image is detected as a straight line, there should be no significant difference in the number of votes for a predetermined number of points. Nevertheless, if the ratio of the vote count to the highest-ranked vote point includes a point that is less than the allowable value, it is highly likely that a long straight line included in the graphic part or photo part in the image is detected. . Therefore, in such a case, detection with low reliability can be avoided by terminating the detection of the tilt angle.

Block diagram of the MFP of the embodiment Block diagram of the pre-processing unit in the embodiment The figure which shows the pre-processing algorithm in an Example Block diagram of Hough transform unit to image rotation unit in the embodiment The figure which shows typically the voting table in an Example. The figure which shows the determination algorithm of the inclination-angle in an Example The figure which shows the determination principle of the inclination angle in an Example The figure which shows the example which binarized the input image directly The figure which shows the reduced image of the input image in an Example The figure which shows the image which removed the noise from the reduction image in an Example. The figure which shows the edge extracted from the image after a noise removal in an Example. The figure which shows the edge extraction filter which was used in the execution example The figure which shows the image which binarized the image of an edge in an Example The figure which shows the image which removed the isolated point from the binarized image in an Example. The figure which shows the image which performed the reduction process after removing an isolated point in an Example. Illustration showing fake maximum voting line The figure which shows the condition where two or more neighborhood points in a voting table are selected as an effective line In an Example, it is a figure which shows the extraction condition of an effective line when masking the vicinity of the extracted effective line, and the lightness shows the number of votes in a voting table The figure which shows typically the extracted effective line in an Example The figure which shows typically the determination method of the inclination-angle in an Example The figure which shows typically the evaluation function of the consistency at the time of extracting the candidate of an inclination angle in an Example.

  In the following, an optimum embodiment for carrying out the present invention will be shown.

  1 to 21 show an embodiment of the multifunction machine 2 as an example. Reference numeral 4 denotes a LAN, 6 denotes a personal computer, which is an example of an information processing apparatus, and 8 denotes a router, which connects the Internet 10 to the MFP 2 and the personal computer 6. The detection and correction of the tilt angle in the embodiment is performed by the multi-function device 2, but may be performed by the personal computer 6 or the like. For example, an image read by the scanner 20 of the multi-function device 2 is transferred to the personal computer 6 and personal The computer 6 may detect and correct the tilt angle.

  The structure of the multifunction device 2 will be described. 12 is a LAN interface, 14 is a bus, 16 is an image memory, and stores image data as a page memory. The memory 18 is a general-purpose memory and stores programs, intermediate data, and the like. The scanner 20 reads an image from original paper in monochrome gray scale or full color, the printer 22 prints the image, and the G3 facsimile unit 24 performs G3 facsimile transmission and reception of the image. The e-mail unit 26 transmits and receives an image as an e-mail attachment file, and the network server unit 28 operates the multi-function device 2 as a document management server, a remote printer, or the like. The user interface 30 accepts user manual input and the like, and the image processing unit 32 performs various image processing.

  In order to detect the tilt angle of the scanned image, the received image, or the accumulated image, a pre-processing unit 40, a Hough transform unit 50, and a tilt angle detection unit 60 are provided. Then, the image is rotated by the image rotation unit 70 by the same angle opposite to the detected inclination angle, and the inclination is corrected.

FIG. 2 shows the configuration of the preprocessing unit 40. Image data is input from the image memory 16 to the reduction processing unit 41, and the resolution of the image is reduced to 1 / n (n is a natural number of 2 or more, preferably 4 to 8, particularly 4 to 6). As a result, the amount of image data is reduced to 1 / n ^2. For reduction, for example, an average value of pixel values may be obtained for each block of n × n pixels. Further, more simply, the reduction processing unit 41 may output the data of one pixel of the representative point in the block of n × n pixels, for example, the upper left vertex of the block as it is. For this reason, the Hough transform is n ² times faster.

  Even if the content and resolution of the data are the same for the same image data, if the reduction rate in the reduction processing unit 41 is different, the detection result of the tilt angle may be different. On the other hand, when the same document is read at different resolutions such as 400 dpi, 600 dpi, and 800 dpi, and reduced to a certain resolution, for example, 100 dpi, the detection result of the tilt angle is common. Therefore, regardless of the resolution at the time of scanning, if the resolution is reduced to a certain resolution by the reduction processing unit 41, the difference in the detected value of the tilt angle due to the difference in resolution can be eliminated.

  If the resolution at the time of scanning is known, such as received data of a facsimile, other than the image scanned by the multifunction device 2, the image is similarly reduced to a constant resolution. The scanning resolution is often determined from the communication protocol or the header of the image file. For this reason, if the resolution at the time of scanning is different, the reduction rate is different, and the resolution of the reduced image is made constant. When the resolution is not described in the header as in the case of a jpeg image, the image is reduced at a constant reduction rate such as 1/4 or 1/8.

  The smoothing unit 42 removes noise from the reduced image using a smoothing filter or the like, thereby preventing random data in the image from being extracted as a straight line by Hough transform. Next, the edge extraction unit 43 extracts edges from the smoothed image using an edge extraction filter such as a Laplacian filter or a Sobel filter. As a result, the influence of the ground color is eliminated, and an edge corresponding to the outline of the character can be extracted even with a white character. Furthermore, the extraction of edges reduces the amount of image data and facilitates Hough transform.

  The binarization unit 44 binarizes the extracted edge. The binarization threshold value may be constant, or may be determined from the brightness at the extracted edge or the distribution of color values. Furthermore, it is not necessary to binarize all edges. For example, when a character part, an image part, and a graphic part in an image can be discriminated, it is preferable to binarize only the character part.

  The image data to be handled is a monochrome gray scale image in the embodiment, but may be an RGB image or the like. In this case, the RGB data may be converted into a brightness image to detect the inclination angle, or the inclination angle may be detected for each of R, G, and B components, and the average value thereof may be used.

  The isolated point removing unit 45 may remove isolated points from the binarized image and remove isolated points from the edge image before binarization. Removal of isolated points reduces the amount of data and reduces the probability of detecting false tilt angles. It should be noted that isolated points are particularly many in a photographic image and are data unrelated to the original inclination of the image.

  The reduction processing unit 46 performs a reduction process on the binarized image after the isolated points are removed. Here, the degeneracy process is a process of cutting the effective pixels when the effective pixels are continuously arranged in the vertical and horizontal directions and the diagonal direction. The effective pixel means a pixel with data in the binarized image, and it does not matter whether it is white or black. The line segment becomes, for example, one point by the reduction process. Circles, squares, and crosses are also one point. As a result, the diagonal lines of the graphic part and the photographic part in the image are reduced to, for example, one point. On the other hand, the character also becomes small data by degeneration, but the straight line formed by the character line is not lost. Since the reduction simplifies the ruled line and the underline to one point, the reduction processing unit 46 may not be provided when the ruled line and the underline are used for the inclination angle detection.

FIG. 3 shows a control algorithm of the pre-processing unit 40. When the personal computer 6 is operated according to this algorithm, the present invention can be implemented. In step 1, the image is read by the scanner 20, and in step 2, the image is reduced to a fixed resolution, thereby improving the processing speed of the Hough transform by n ² times. In step 3, noise is reduced by smoothing, and in step 4, edges are extracted to remove the influence of the ground color, etc., and white characters and the like can be detected. Then, binarization is performed in step 5, and isolated points are removed in step 6. This can reduce the influence of an isolated point such as a photographic image on the detection of the tilt angle. The removal of isolated points may be performed before binarization. Preferably, in step 7, the data is simplified by a reduction process, and in particular, a straight line of a photograph part or a graphic part is simplified to, for example, one point.

  The preprocessing of Step 2 to Step 7 is performed in synchronization with image scanning, and the Hough conversion in the Hough conversion unit 50, the detection of the inclination angle in the inclination angle detection unit 60, and the inclination angle in the image rotation unit 70 are performed. Corrections can be made in synchronization with the scan. Here, “synchronization” means that these processing speeds are equal to or higher than the reading speed of the scanner 20, and in this way, the inclination of the image can be corrected in real time simultaneously with the scanning of the image.

  FIG. 4 shows the structure of the Hough transform unit 50 to the image rotation unit 70. The Hough transform unit 50 includes, for example, 16 to 32 execution units 51, and executes the Hough transform in parallel. The execution unit 51 includes a ρ value calculation unit 52 and a table 53 for sin and cos. The range of angles stored in one table 53 is, for example, 5.625 ° (180/32) so that 32 tables 53 can cover 0 ° to 180 °, and the angle increment is, for example, 0.176. ° (180 ° / 1024). The ρ value calculation unit 52 reads the values of sin θ and cos θ from the table 53, and sets the ρ value to ρ = x cos θ + y sin θ for the effective pixel position (x, y) output from the degeneration processing unit 46 or the isolated point removal unit 45. calculate. The voting table 54 is a two-dimensional table of θ and ρ, and the management unit 55 manages a plurality of execution units 51.

  FIG. 5 shows the relationship between the ρ value calculation unit 52 and the voting table 54. Each point in the table 54 represents a combination of ρ and θ, which represents a line segment. Assume that ρ and θ are each divided into 1024 levels, and when x and y values are input, the value of ρ is calculated for each θ and 1 is added to the data at the corresponding position in the table. In addition, θ decomposes the range of 180 ° into, for example, 256 to 4096 level, preferably 1024 to 2048 level, and the inclination angle is about 0.05 ° to 0.3 °, preferably about 0.1 ° to 0.2 °. Make detection with accuracy. Further, the resolution of ρ may be lower than the resolution of θ. The execution unit 51 has, for example, 32 units, and each ρ value calculation unit 52 takes charge of θ for 32 levels (1024/32). As a result, for example, the (ρ, θ) value is voted as indicated by a circle in the third column from the left. Reference numeral 57 denotes a mask, which masks the periphery of a point already extracted as an effective line (extracted straight line) from the table 54 so that the next effective line is extracted from the outside of the mask 57.

  Returning to FIG. 4, the effective line extraction unit 61 extracts, for example, the upper m-th point from the voting table 54. A point with a large number of votes in the voting table 54 is likely to be an effective line in the image. Then, a mask 57 is applied around the extracted points so that other points are not extracted from the mask 57. The candidate angle extraction unit 62 extracts inclination angle candidates (k, for example, m / 2 or more) from the extracted effective lines, and the determination unit 63 determines the inclination angle θd based on these candidate angles. The determined inclination angle θd is input to the coordinate calculation unit 71, and the coordinates (address) after inclination correction are calculated for each pixel of the image stored in the image memory 16. The address generation unit 72 generates a read address to the image memory 16 based on the coordinates before tilt correction. The bicubic processing unit 73 interpolates the read data, and writes the interpolated data back to the image memory 16 based on the coordinates after the inclination correction. By these processes, the image in the image memory 16 is rotated so as to correct the inclination. A simple affine transformation or the like may be used for the rotation.

  FIG. 6 shows an algorithm of the tilt angle detection unit 60, and points from the voting table consisting of the upper m-th place, for example, the 16th or eighth place (ρ, θ) are extracted (step 11). Then, the vicinity of the extracted points, for example, the vicinity of a square having 5 to 40 levels on one side is masked (step 12), so that two or more points are not extracted from the same vicinity. It should be noted that only (ρ, θ) having a vote count of ½ or more of the maximum vote count is extracted. Therefore, if the number of votes at the mth point is less than ½ of the maximum number of votes, an error is detected, the inclination angle is not detected, and the image rotation unit 70 does not rotate the image (step 13). Steps 12 and 13 are auxiliary processes for executing Step 11. In this way, m points that are not in the vicinity of other points and that are 1/2 or more of the maximum number of votes are extracted. 1/2 or more is an example. For example, it may be 2/3 or more or 0.4 or more. Steps 11 to 13 correspond to the processing of the effective line extraction unit 61.

  Next, an angle for obtaining a tilt angle candidate is θx, and its orthogonal angle is φx. Here, φx is positive or 0, and φx = θx + π / 2 or φx = θx−π / 2. For example, δ is 1 ° or 0.5 °, and θx is determined so that the number of points (ρ, θ) within the range of θx ± δ and φx ± δ is maximized. . Then, points that fall within the ranges of θx ± δ and φx ± δ are output (step 14). In other words, the angle at which the number of points having the same angle θ component in the range of ± δ is the maximum for m points is defined as θx. Instead of simply considering the number of points where the θ components coincide within the range of ± δ, the sum of the votes may be maximized for the points falling within the range of θx ± δ and φx ± δ. . If the number k of these points is less than the threshold, for example, less than m / 2 or less than m / 3, an error is determined in step 15. In the case of an error, the inclination angle is not detected, and the image rotation unit 70 does not rotate the image. Steps 14 and 15 correspond to the processing of the candidate angle extraction unit 62.

  Step 16 corresponds to the processing of the determination unit 63 and outputs, for example, a simple average value of θ as the inclination angle θd for the k points (ρ, θ) obtained in step 14. Alternatively, a weighted average of θ at k points may be used, and the number of votes vi may be used as the weight. Instead of these, θ having the largest number of votes vi among the k θ candidates θ1 to θk may be used as the inclination angle θd. In step 16, θd is output. If θd exceeds π / 2, θd−π / 2, that is, φd is output.

  7 to 20 show the operation of the embodiment. The upper part of FIG. 7 shows a document image, and the inclination of the character string in the image is detected as a straight line. Therefore, the object of the embodiment is to convert the upper image in FIG. 7 into the lower image in FIG. 7 and perform Hough transform on the image.

FIG. 8 shows an example in which an input image (image in the image memory 16) is directly binarized without performing reduction, smoothing, or edge extraction. As marked with a circle, black pixels such as a photo portion other than characters are shown. The number of (effective pixels in this case) increases, the burden of Hough conversion increases, and the influence of oblique lines in the photograph becomes stronger, resulting in a decrease in detection accuracy. Therefore, the input image is handled in gray scale and subjected to image processing such as reduction, smoothing, and edge extraction, and then binarized. In order to perform the Hough transform at high speed, the input image is reduced to 1 / n, thereby reducing the subsequent calculation time to 1 / n ² (FIG. 9).

  Next, noise is removed by smoothing (FIG. 10), and an edge is extracted to obtain the image of FIG. By extracting the edge, the influence of the ground color is eliminated, and it is possible to cope with white characters, and further simplifies the data and facilitates the Hough conversion.

  FIG. 12 shows an edge extraction filter 80. By applying the filter 80 from the upper left side to the processed image 81, the lower and right edges 82 of the processed image 81 are extracted. As a result, the amount of edges can be halved compared to the case of detecting the entire edge of the image 81 to be processed. As described above, preferably, two types of edges are extracted from among the four types of edges, top, bottom, left, right, bottom and left, top and right, and top and left.

  When the image after edge extraction is binarized, the image of FIG. 13 is obtained. When the isolated points are removed from the image of FIG. 13, the image of FIG. 14 is obtained. Note that the removal of isolated points may be performed on the edge image of FIG. As shown in FIGS. 13 and 14, if a graphic part line or a photographic part line remains, it may cause an erroneous detection of the tilt angle. Therefore, when the reduction process is performed, the image shown in FIG. 15 is obtained. The lines in the photograph or the figure are converted into simple dots or short line segments, and the lines of character strings are basically stored.

  Therefore, by performing Hough transform on the image after the reduction processing, only a straight line composed of character lines can be detected. However, there are cases where it is desired to use the direction of the ruled line for detecting the inclination of the image with the ruled line. In this case, the degeneration process is not necessary.

  If data is accumulated in the voting table by the Hough transform and only the point with the largest number of votes is simply extracted from the voting table, erroneous detection may be performed. FIG. 16 shows such an example. In FIG. 16, only a part of the image is displayed. Therefore, it is difficult to know where the “maximum voting line” is extracted, but it is extracted from a photograph part or the like in the image. is there. Further, even if it is not an extreme example as shown in FIG. 16, there is a possibility that the maximum vote may be made on the solid line in the graphic part at the bottom of FIG. 13 and FIG. The horizontal line in the lower graphic part of FIG. 13 is parallel to the inclination of the image, but this is coincidental. Therefore, a plurality of points with the highest number of votes are extracted so as not to extract a straight line that does not correspond to the original inclination of the image. If points up to the top m are simply extracted here, a plurality of neighboring points in the voting table are often extracted. Therefore, the highest point is extracted from the voting table 54, and the vicinity of the extracted point is extracted by masking the vicinity of the extracted point so as not to extract the point near the extracted point. In this way, the upper m-th point is extracted. If the lowest point of the extracted top m ranks does not have, for example, ½ or more votes of the highest point, the detection is regarded as an error. When a line consisting of characters is extracted, there should be no significant difference in the number of votes between the maximum number of votes and the m-th place from the top. Despite this, there is a great difference that the highest point is likely to be a straight line portion in the figure. Alternatively, from the top m ranks (for example, 16th rank), points having a voter number of a predetermined ratio or more (for example, 1/2 or more) with respect to the highest point are extracted, and the number of extracted points is predetermined. When the value is equal to or less than k (for example, 4), the tilt angle may not be detected as an error.

  FIG. 18 schematically shows the situation of the voting table. Here, the number of votes is represented by brightness. The vicinity of the selected point is covered with a black square mask. In FIG. 18, the top 8 points have been extracted and each is masked. The shape of the mask is arbitrary, and it is to avoid extracting a straight line whose direction is close to that of the extracted straight line (a voting table point) and whose distance is small. As a result, for example, the number of parallel lines extracted in the photograph portions of FIGS. 13 and 14 can be reduced. Also, even if there is no figure or photo and only a straight line consisting of character lines can be extracted, the original can be extracted without extracting lines from a specific area by extracting distant lines without extracting adjacent lines. Allows extracting rows from the whole.

  An inclination angle is determined for the extracted upper m-th point. This procedure is schematically shown in FIGS. 19 and 20, where m is, for example, 8. When a character row is detected, the upper and lower columns of the character block can be detected at the same time, so that the inclination angle θd and the orthogonal angle φd are significant. In addition, the tolerance for extracting candidates is δ (about 0.5 ° or 1 °), and θx is determined so that the number k of points that fall within the range of angles θx ± δ and φx ± δ is maximized. To do. Here, if the value of k is less than a threshold value (for example, m / 2) for any angle, an error is assumed, for example. When an angle θx including a point equal to or greater than the threshold is obtained, the candidate angle is determined by a simple addition average or a weighted average of the angles of the included points. This is the process of step 16 in FIG.

  When the inclination angle θd is determined with an accuracy of, for example, about ± 0.1 ° as described above, the image is rotated by the image rotation unit 70 to correct the inclination. The image rotation itself is a simple process and can be executed in synchronization with, for example, scanning.

FIG. 21 shows the consistency evaluation functions 101 and 102. The horizontal axis represents the deviation from the extraction angle θx, and the vertical axis represents the weights of the evaluation functions 101 and 102. Reference numerals 105 to 110 denote the extracted data by angles, and the height of the data 105 to 110 is the number of votes. In the embodiment, the evaluation function 101 having a weight of 0/1 is used. However, as with the evaluation function 102, the weight may be reduced in accordance with the deviation from θx. For both evaluation functions 101 and 102,
Consistency may be evaluated by reflecting the number of votes, such as the sum of those included in the evaluation function (weight of data 105 to 110 × number of votes of each data).

  In the process of FIG. 6, the inclination angle θd is determined by the simple average or the weighted average of the angle θ for the data 105 to 109 having a weight of 1 that has passed the evaluation function 101. However, instead of using the number of votes as it is in the weighted average, the square root or the square of the number of votes may be used, or the weighted average reflecting the number of votes may be used so that the weight increases as the number of votes increases.

  In the embodiment, θx is determined first, data 105-109 and the like are extracted, and then the angle θd is determined using the data 105-109. Therefore, the data 105 to 109 can be sufficiently reflected in the determination of the angle θd. However, more simply, the angle θx may be used as it is as the inclination angle θd. However, if the angle θx is used as it is as the inclination angle θd, the angle θx is determined so as to include the data 109 at the end, and therefore the inclination angle tends to be easily influenced by the data 109 at the end. Therefore, it is preferable to use two stages, that is, determination of the angle θx and determination of the inclination angle θd as in the embodiment.

In the embodiment, the example in which the rotation and the correction of the tilt angle are performed inside the multi function device 2 has been described. However, it may be performed by the personal computer 6 or the like. In that case, when the image processing program of the embodiment is executed by an information processing apparatus such as a personal computer, the information processing apparatus becomes the image processing apparatus of the present invention.
When detecting the inclination of a book spread, for example, if it is input that the type of the document is a page spread before scanning, separate left and right inclination angles can be detected. Alternatively, the input image may be divided into right and left to obtain an inclination angle, and if the left and right inclination angles are different, it may be estimated that the image is a spread image.

2 MFP 4 LAN
6 personal computer 8 router 10 internet 12 LAN interface 14 bus 16 image memory 18 memory 20 scanner 22 printer 24 G3 facsimile unit 26 e-mail unit 28 network server unit 30 user interface 32 image processing unit 40 preprocessing unit 41 reduction processing unit 42 Smoothing unit 43 Edge extraction unit 44 Binarization unit 45 Isolated point removal unit 46 Degeneration processing unit 50 Hough transform unit 51 Execution unit 52 ρ value calculation unit 53 Table 54 Voting table 55 Management unit 56 Data 57 Mask 60 Inclination angle detection unit 61 Effective line extraction unit 62 Candidate angle extraction unit 63 Determination unit
70 image rotation unit 71 coordinate calculation unit 72 address generation unit 73 bicubic processing unit 80 filter 81 processed image 82 edge 101, 102 evaluation function 105-110 data

Claims (11)

An image having a Hough transform unit for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as headings In the processing device,
Extracting a predetermined number of points with the highest number of votes from the vote table;
The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx ,
Furthermore, when extracting the predetermined number of points, the neighborhood of the extracted points in the voting table is masked, and other points within the masked range are not extracted from the voting table. Processing equipment. An image having a Hough transform unit for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as headings In the processing device,
  Extracting a predetermined number of points with the highest number of votes from the vote table;
  The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx,
  Furthermore, when the ratio between the number of points within the allowable range from the extraction angle θx within the predetermined number of points and the predetermined number is less than a predetermined value, the inclination angle of the image is detected. An image processing apparatus configured so as not to be performed. An image having a Hough transform unit for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as headings In the processing device,
Extracting a predetermined number of points with the highest number of votes from the vote table;
The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx,
Further, the image processing apparatus is configured not to detect an inclination angle of the image when the predetermined number of points includes a point whose ratio of the number of votes with the highest number of votes is less than an allowable value. . Among the predetermined number of points, the inclination angle of the image is determined by an average of the angles θ with respect to a point that includes the angle θ within an allowable range from the extraction angle θx. The image processing apparatus according to claim 1 . Of the predetermined number of points, the inclination angle of the image is determined by an average reflecting a weight corresponding to the number of votes of the angle θ with respect to a point that includes the angle θ within the allowable range from the extraction angle θx. The image processing apparatus according to claim 1, wherein the image processing apparatus is configured as described above. In an image processing method for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as a headline,
  Extracting a predetermined number of points with the highest number of votes from the voting table, masking the neighborhood of the extracted points in the voting table so that other points in the masked range are not extracted from the voting table,
  Further, the angle θx for extraction is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined angle θx for extraction, Image processing method. In an image processing method for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as a headline,
  Extracting a predetermined number of points with the highest number of votes from the vote table;
  The angle θx for extraction is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined angle θx for extraction,
  Furthermore, when the ratio between the number of points within the allowable range from the extraction angle θx within the predetermined number of points and the predetermined number is less than a predetermined value, the inclination angle of the image is detected. An image processing method characterized by not performing the processing. In an image processing method for extracting a straight line in an image by converting the coordinates of a point in the image into a distance ρ and an angle θ and voting on a voting table having the distance ρ and the angle θ as a headline,
Extracting a predetermined number of points with the highest number of votes from the vote table;
The extraction angle θx is determined so that the consistency with the angle θ of the predetermined number of points is maximized, and the inclination angle of the image is obtained based on the determined extraction angle θx.
Further, the image processing method is characterized in that the inclination angle of the image is not detected when the predetermined number of points includes a point where the ratio of the vote number to the highest vote point is less than an allowable value . Read into the information processing apparatus,
  A function for converting the coordinates of a point in the image into a distance ρ and an angle θ, and voting on a voting table having the distance ρ and the angle θ as headlines;
  A function of extracting a predetermined number of points with the highest number of votes from the voting table, masking the neighborhood of the extracted points in the voting table, and preventing other points within the masked range from being extracted from the voting table When,
  Further, the function of determining the angle θx for extraction so that the consistency with the angle θ of the predetermined number of points is maximized, and obtaining the tilt angle of the image based on the determined angle θx for extraction is executed. An image processing program. Read into the information processing apparatus,
A function for converting the coordinates of a point in the image into a distance ρ and an angle θ, and voting on a voting table having the distance ρ and the angle θ as headlines;
A function of extracting a predetermined number of points with the highest number of votes from the vote table;
A function of determining the angle of extraction θx so as to maximize the consistency with the angle θ of the predetermined number of points, and determining the inclination angle of the image based on the determined angle of extraction θx;
The inclination angle of the image is not detected when the ratio between the number of points within the allowable range from the extraction angle θx and the predetermined number is less than a predetermined value among the predetermined number of points. An image processing program for executing the function to Read into the information processing apparatus,
  A function for converting the coordinates of a point in the image into a distance ρ and an angle θ, and voting on a voting table having the distance ρ and the angle θ as headlines;
  A function of extracting a predetermined number of points with the highest number of votes from the vote table;
  A function of determining the angle of extraction θx so as to maximize the consistency with the angle θ of the predetermined number of points, and determining the inclination angle of the image based on the determined angle of extraction θx;
  Further, an image processing program for executing a function of preventing the inclination angle of an image from being detected when the predetermined number of points includes a point with a ratio of the number of votes with the highest number of votes being less than an allowable value. .