JP4565396B2 - Image processing apparatus and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing program, and more particularly to an image processing apparatus and an image processing program for detecting an inclination angle of a document image read in an inclined manner when reading an image and correcting the inclination.

  Analyzes the image data obtained when the document image is read by the image scanner and the image data received by the facsimile machine, extracts the region from the document image, and determines the type and attribute of the image included in the document. A technique is known in which character recognition processing is performed on an area determined as a character, and a character or a sentence is extracted from a document read as an image.

  In the field of this type of image processing, in order to correctly execute processing such as segmentation and character recognition, it is premised that the document image read by the image scanner or the facsimile machine on the transmission side has no inclination. Therefore, when the read image has a tilt, as a pre-process for processing such as segmentation and character recognition, the tilt angle of the image (hereinafter referred to as “skew angle”) is detected and the tilt is detected. It is necessary to perform so-called skew correction.

  Conventionally, a technique using Hough transformation is known as a technique for detecting a skew angle of a document image and performing skew correction based on the detection result (see, for example, Patent Document 1). In the prior art described in Patent Document 1, the following processing is performed when detecting the skew angle.

  First, a filtering process is performed on an input image, and a binarization process is performed on an image whose density difference is enhanced by the filtering process to generate a binary image. Next, a Hough transform is performed on each pixel of the generated binary image, a histogram is created in the Hough space, and coordinates whose frequency is equal to or higher than a predetermined threshold in the Hough space are extracted. Then, after the extracted coordinates are grouped, representative point coordinates are extracted for each group, and the skew angle of the image is estimated from the extracted coordinates.

  Further, in Patent Document 1, after extracting coordinates having a frequency equal to or higher than a predetermined threshold in the Hough space, the number of extracted coordinates is integrated for each angle to generate a histogram, and the angle at which the frequency is maximum is obtained. A technique for estimating the skew angle of an image is also described.

JP 11-328408 A

  However, since the above-described conventional technology adopts a configuration in which the Hough conversion process and the histogram generation process are performed on the entire document image, ON pixels (for example, black pixels with a white background) are included in the binarized image. If the number of pixels is large, the calculation process during the Hough transform process takes time, and there is a problem that the processing speed of the skew angle detection becomes slow. Specifically, in the case of an image with a small number of ON pixels, for example, a reduced image, even if the Hough transform process is performed on the entire image, the impact on the processing speed of the skew angle detection is not so great, but the original resolution image In this case, since the number of ON pixels is large and the calculation process takes time, the impact on the processing speed of the skew angle detection is large.

  Also, by performing the Hough transform process on the entire image, the skew angle detection process is performed even on useless ON pixels. In some cases, an erroneous detection of the skew angle may occur. There is also.

  In order to improve the processing speed of the skew angle detection, a method of reducing the number of ON pixels by extracting the contour of the binary image and detecting the skew angle based on the coordinates of the contour pixel can be considered. However, if this method is adopted, the processing time of the calculation can be shortened by reducing the number of ON pixels, so that the processing speed of the skew angle detection can be improved, but the detection accuracy is reduced by reducing the number of ON pixels to be calculated. There is a problem of doing.

  The present invention has been made in view of the above problems, and an object of the present invention is to perform skew angle detection without reducing detection accuracy in processing for detecting a skew angle by a technique using Hough transform. An object of the present invention is to provide an image processing apparatus and an image processing program capable of improving the processing speed.

  In order to achieve the above object, the present invention adopts the following configuration. That is, the same label is assigned to consecutive pixels in the binary image obtained by binarizing the input image, and among the consecutive pixel groups to which the same label is assigned, the continuous pixel group whose feature amount satisfies a predetermined criterion is specified. Extract as a pixel group. Then, angle detection is performed on the extracted specific pixel group by Hough transform, and skew correction is performed on the input image based on the detected angle (skew angle).

  In image processing that detects and corrects skew angle, the most suitable, simplest, and most accurate figure for detecting the skew angle of an input image using a technique using Hough transform is, for example, a long straight line, an elongated rectangle, Or it is a large rectangle. Therefore, by continuously setting a predetermined reference to be extracted as the specific pixel group, it is possible to extract a continuous pixel group that forms a figure such as a straight line, an elongated rectangle, or a large rectangle as the specific pixel group. The extracted specific pixel group is subjected to angle detection by Hough transform to detect a skew angle, and is a continuous pixel group that forms the most suitable, simplest, and most accurate figure for skew angle detection. The Hough transform process is performed only for.

  In order to achieve the above object, the present invention extracts a pixel indicating a contour of a binary image obtained by binarizing the input image, and performs a Hough transform process on the binary image data obtained by extracting the contour. The xy plane is detected based on one or more points that are frequently detected in the polar coordinate space by the Hough transform process in the angle detection process in the previous stage, while detecting the tilt angle in stages. The useful pixel group above is extracted and passed to the subsequent angle detection process. Then, the skew correction is performed on the input image based on the angle (skew angle) detected by the angle detection process at the final stage.

  In the skew angle detection / correction process described above, after the Hough transform process is performed on the binary image data extracted from the contour, the xy plane is based on one or more points that are frequently used in the polar coordinate space by the Hough transform process. By extracting the above useful pixel group, it is possible to extract a pixel group that forms a figure such as a straight line, an elongated rectangle, or a large rectangle as the useful pixel group. Then, by detecting the skew angle by performing angle detection by the Hough transform on the extracted pixel group, the pixel group that forms the most suitable, simplest, and most accurate figure for the skew angle is detected. Only the Hough conversion process is performed.

  According to the present invention, the Hough transform process is performed not on the entire image but on a continuous pixel group (specific pixel group) that forms the most suitable, simplest, and most accurate figure for detecting a skew angle. As a result, the skew angle can be detected with high accuracy, and the number of pixels subject to calculation processing can be minimized. It can be improved.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of an image processing system including an image processing apparatus according to the present invention. As shown in FIG. 1, the image processing system includes an image data input unit 10, an image processing unit 20, and an image data output unit 30.

  The image data input unit 10 includes an image reading device such as an image scanner, reads an image of a document original, and inputs the image data to the image processing unit 20. In the case of adopting a system configuration in which image data read by an image reading device such as an image scanner is temporarily stored in a storage device such as a server, the server functions as the image data input unit 10. In the facsimile apparatus, the receiving facsimile apparatus functions as the image data input unit 10.

  The image processing unit 20 corresponds to the image processing apparatus according to the present invention, detects a skew angle (tilt angle) of an image (input image) based on the image data input from the image data input unit 10, and based on the detected angle. In addition, skew correction is performed on the image data.

  The image processing unit 20 includes a CPU (Central Processing Unit) 21, an I / O circuit 22, a ROM 23, a RAM 24, an HDD (Hard Disk Drive) device 25, and the like, and these components are connected via a bus line 26. It is the structure connected so that mutual communication was possible.

  The CPU 21 controls processing of the entire image processing unit 20 including calculation processing. The I / O circuit 22 manages input / output with peripheral devices including the image data input unit 10 and the image data output unit 30. The ROM 23 stores processing programs for various processes executed under the control of the CPU 21. The RAM 24 is a primary storage device used when executing various processes. The HDD 25 stores image data processed under the control of the CPU 21 and image data captured from the outside.

  The image data output unit 30 outputs the image data whose skew has been corrected by the image processing unit 20 to an image processing device (not shown) such as a character recognition device in the next stage. The image data output unit 30 includes an output device such as a printer or a display and its control means, and prints out an image based on the image data skew-corrected by the image processing unit 20 on printing (recording) paper, or It is also possible to display on the display screen.

[First Embodiment]
FIG. 2 is a block diagram illustrating a functional configuration example of the image processing apparatus (image processing unit) 20A according to the first embodiment of the present invention.

  As shown in FIG. 2, the image processing apparatus 20A according to the present embodiment includes a binarization unit 201, a labeling unit 202, a specific pixel group extraction unit 203, a coordinate extraction unit 204, a Hough conversion unit 205, and an angle calculation. The configuration includes a unit 206 and a skew correction unit 207.

  The binarization unit 201 converts the image data input from the image data input unit 10 in FIG. 1 and representing the shading with a plurality of bits into foregrounds such as characters, lines, patterns, and photographs included in the input image. Each pixel value is 1 when the pixel belonging to is an ON pixel (for example, a black pixel) with a logical value of “1” and the pixel belonging to the background area is an OFF pixel (for example, a white pixel) with a logical value of “0”. The image is binarized into bit binary image data and supplied to the labeling unit 202.

  The labeling unit 202 executes a labeling process for assigning the same label to consecutive pixels (specifically, ON pixels) in the binary image (data) binarized by the binarization unit 201. Hereinafter, a set of continuous pixels assigned the same label is referred to as a continuous pixel group.

  The specific pixel group extraction unit 203 extracts, as the specific pixel group, a continuous pixel group whose feature amount satisfies a predetermined criterion in the continuous pixel group to which the same label is given by the labeling process in the labeling unit 202. Here, the feature amount of the continuous pixel group is a circumscribed rectangle size (vertical / horizontal length and aspect ratio) of the continuous pixel group, the number of pixels of the continuous pixel group, and the like.

  As a specific pixel group extraction method in the specific pixel group extraction unit 203, the following method can be considered. However, these are merely examples, and the present invention is not limited to these.

Method 1: A continuous pixel group having a size of a circumscribed rectangle of the continuous pixel group (vertical / horizontal length and rectangular area) having a predetermined size or more is extracted as a specific pixel group.
Method 2: A continuous pixel group having the largest size (vertical / horizontal length and rectangular area) is extracted as a specific pixel group among circumscribed rectangles of the continuous pixel group.
Method 3: A continuous pixel group having a predetermined number of pixels (including the above) is extracted as a specific pixel group.
Method 4: The continuous pixel group having the largest number of pixels among the continuous pixel groups is extracted as the specific pixel group.

Method 5: A linear pixel is calculated from the size (vertical / horizontal length and aspect ratio) of the circumscribed rectangle of the continuous pixel group and the number of pixels of the continuous pixel group, and the linear pixel satisfies the predetermined criterion. Are extracted as a specific pixel group.
Method 6: The linearity is calculated from the size (vertical / horizontal length and aspect ratio) of the circumscribed rectangle of the continuous pixel group and the number of pixels of the continuous pixel group, and the continuous pixel group having the highest linearity is extracted as the specific pixel group To do.
Method 7: The linearity is calculated from the size (vertical / horizontal length and aspect ratio) of the circumscribed rectangle of the continuous pixel group and the number of pixels of the continuous pixel group, and the linearity is the highest and the linearity is determined in advance. A continuous pixel group that satisfies the specified criteria is extracted as a specific pixel group.

Method 8: A rectangular pixel is calculated from the size (vertical / horizontal length and aspect ratio) of the circumscribed rectangle of the continuous pixel group and the number of pixels in the continuous pixel group, and the continuous pixel group satisfies the predetermined criterion. Are extracted as a specific pixel group.
Method 9: Calculate the rectangularity from the size (vertical / horizontal length and aspect ratio) of the circumscribed rectangle of the continuous pixel group and the number of pixels of the continuous pixel group, and extract the continuous pixel group having the highest rectangularity as the specific pixel group To do.
Method 10: The rectangularity is calculated from the size of the circumscribed rectangle of the continuous pixel group (vertical / horizontal length and aspect ratio) and the number of pixels of the continuous pixel group, and the rectangularity is the highest and the rectangularity is determined in advance. A continuous pixel group that satisfies the specified criteria is extracted as a specific pixel group.

  By extracting a specific pixel group using such various methods, it is most suitable, simplest, and most accurate to detect the skew angle of the input image by a technique using Hough transform as the specific pixel group. It is possible to extract a continuous pixel group that forms a high figure, for example, a straight line, an elongated rectangle, or a large rectangle.

  As an example, in the case of a document image in which lines and characters are mixed, such as a drawing created by CAD (Computer Aided Design), extraction of a specific pixel group is performed on the document image using the method 7 described above, for example. By doing so, it is possible to extract only a straight line from a document image in which lines and characters are mixed. As is apparent from the principle of the Hough transform described later, the straight line can be said to be the most suitable, simplest and most accurate figure for detecting the skew angle of the input image by the technique using the Hough transform.

  In the specific pixel group in the specific pixel group extraction unit 203, the specific pixel group included in the extracted specific pixel group (graphic) is excluded from the extraction target due to the graphic inclusion relationship.

  The coordinate extraction unit 204 extracts the coordinates of each ON pixel of the continuous pixel group extracted as the specific pixel group by the specific pixel group extraction unit 203.

The Hough conversion unit 205 performs a well-known Hough conversion process on the continuous pixel group whose coordinates have been extracted by the coordinate extraction unit 204. Here, the Hough conversion process will be described. With respect to a straight line passing through the point (x, y) on the xy coordinates, if a perpendicular is drawn from the origin to this straight line, the length is ρ, and the angle between the perpendicular and the x axis is θ, the straight line is ρ On the -θ coordinate (on the polar coordinate),
ρ = x cos θ + ysin θ
Is converted into a point (ρ, θ) that satisfies

When the point on the xy coordinate is considered as a collection (intersection) of a plurality of straight lines, the point (x, y) on the xy coordinate is on the ρ-θ coordinate (on the polar coordinate).
ρ = x cos θ + ysin θ (0 ≦ θ <π)
It can be expressed by the following curve formula. This curve is a Hough curve, and the process for obtaining this Hough curve is the Hough transform. That is, when a Hough transformation process is performed on a plurality of points on a straight line and a Hough curve is drawn on the ρ-θ coordinates (on the polar coordinates), these Hough curves intersect at one point.

  Using this, the angle θ of the above equation is sequentially changed from 0 to π for each coordinate of all the ON pixel coordinates extracted by the coordinate extraction unit 204, and this change in the angle θ is dealt with. If the distance ρ obtained in this way is plotted on the ρ-θ coordinate, there will be a number of points where many curves intersect, that is, a number of frequently plotted points. Can be found. The inclination of this straight line can be considered as the inclination of the image.

  Note that the Hough conversion process does not necessarily have to be performed on all the ON pixels of the continuous pixel group whose coordinates have been extracted. In other words, it is possible to provide a thinning processing circuit in the previous stage of the Hough conversion unit 205 so that the ON pixels of the continuous pixel group are thinned out at an appropriate ratio and the remaining ON pixels are subjected to the Hough conversion processing. It is. The thinning-out process can also be performed as a pre-process of the coordinate extraction process.

  The angle calculation unit 206 calculates the inclination angle of the figure (circumscribed rectangle / straight line) formed by the continuous pixel group (specific pixel group) as the skew angle of the input image based on the result of the Hough conversion by the Hough conversion unit 205. . Specifically, the angle calculation unit 206 obtains an angle with reference to θ of coordinates (ρ, θ) having a high frequency on the ρ-θ coordinate subjected to the Hough transform by the Hough transform unit 205, and calculates the angle as x−. Calculated as the skew angle of the image in the y plane. The angle calculation unit 206, together with the Hough conversion unit 205, constitutes the angle detection means in the claims.

  Based on the skew angle of the input image calculated by the angle calculation unit 206, the skew correction unit 207 performs skew correction by rotating the input image so that the skew angle becomes zero. As a method for rotating the image, for example, a known method such as affine transformation can be used.

  Regarding the components of the image processing apparatus 20A configured as described above, that is, the binarization unit 201, the labeling unit 202, the specific pixel group extraction unit 203, the coordinate extraction unit 204, the Hough transform unit 205, the angle calculation unit 206, and the skew correction unit 207. It may be realized by a software configuration using a computer device such as a PC (Personal Computer) that executes each function such as information storage processing, image processing, and arithmetic processing by executing a predetermined program.

  However, the present invention is not limited to the implementation by software configuration, and can also be implemented by a hardware configuration or a combined configuration of hardware and software. When realized by a software configuration, there is a program that causes a computer to function as the binarization unit 201, the labeling unit 202, the specific pixel group extraction unit 203, the coordinate extraction unit 204, the Hough conversion unit 205, the angle calculation unit 206, and the skew correction unit 207. An image processing program according to the present invention is provided.

  It can also be said that the program for executing the processing of each step in the image processing sequence described below is the image processing program according to the present invention. These image processing programs may be installed in the computer in advance. However, it may be provided by being stored in a computer-readable storage medium instead of being installed in advance, or distributed via wired or wireless communication means. .

(Skew angle detection / correction processing)
Next, an example of a procedure of skew angle detection / correction processing according to the first embodiment in which a skew angle of an input image is detected and skew correction is performed based on the detected angle will be described with reference to the flowchart of FIG.

  First, input image data is captured (step S11), and then the captured image data is binarized to generate binary image data (step S12). Next, continuous ON pixels in the binarized image (data). A labeling process for assigning the same label is performed (step S13). Here, a set of continuous pixels to which the same label is assigned becomes a continuous pixel group.

  Next, among the continuous pixel groups to which the same label is assigned using the above-described methods 1 to 10 and the like, the circumscribed rectangle size (vertical / horizontal length and aspect ratio) of the continuous pixel group and the continuous pixel group A continuous pixel group whose feature quantity such as the number of pixels satisfies a predetermined criterion is extracted as a specific pixel group (step S14), and then the coordinates of each ON pixel of the continuous pixel group extracted as the specific pixel group are extracted (step S15).

  Next, the Hough transform process is performed on the coordinate-extracted continuous pixel group (step S16), and then the figure (circumscribed rectangle / straight line) formed by the continuous pixel group (specific pixel group) based on the Hough transform result. The tilt angle is calculated as the skew angle of the input image (step S17). Then, the skew correction is performed by rotating the input image based on the calculated skew angle (step S18), and the series of skew angle detection / correction processing ends.

  By the way, in image processing that detects and corrects skew angle, the most suitable, simplest, and most accurate figure for detecting the skew angle of the input image by the technique using Hough transform is the principle of Hough transform described above. As can be seen from the figure, it can be said to be a figure such as a long straight line, a long and narrow rectangle, or a large rectangle.

  Therefore, as described above, the same label is assigned to consecutive pixels in the binarized image to form a continuous pixel group, and among the plurality of continuous pixel groups, a continuous pixel group whose feature amount satisfies a predetermined criterion is specified pixel. By extracting as a group, continuous pixel groups useful for skew angle detection, for example, the most suitable, simplest and most accurate figure for skew angle detection, specifically long straight lines, elongated rectangles, or larger A pixel group forming a figure such as a rectangle can be extracted. Then, by performing skew angle detection by Hough transform using only the extracted specific pixel group, only the continuous pixel group that forms the most suitable, simplest, and most accurate figure for skew angle detection is used. Since the Hough transform process is performed, the accuracy of detecting the skew angle can be increased.

  In addition, instead of performing the Hough transform process on the entire input image, only the continuous pixel group useful for skew angle detection is extracted as a specific pixel group, and the Hough transform process is performed to effectively contribute to the skew angle detection. Since characters such as non-performing characters, short straight lines, small rectangles and the like can be excluded from the calculation processing target, the number of pixels to be subjected to the calculation processing in the Hough transform can be minimized. As a result, it is possible to reduce the time required for the arithmetic processing in the Hough transform, so that the processing speed of a series of processes for detecting the skew angle and correcting the skew based on the skew angle can be improved.

[Second Embodiment]
FIG. 4 is a block diagram illustrating a functional configuration example of an image processing apparatus (image processing unit) 20B according to the second embodiment of the present invention.

  As shown in FIG. 4, the image processing apparatus 20B according to the present embodiment includes a gray conversion unit 211, a binarization unit 212, a contour extraction unit 213, a reduction unit 214, an approximate angle detection unit 215, and a frequency multipoint. The configuration includes an extraction unit 216, an inverse Hough transform unit 217, an enlargement unit 218, an expansion unit 219, a logical product unit 220, a labeling unit 221, a predetermined feature label extraction unit 222, a detailed angle detection unit 223, and a skew correction unit 224. Yes.

  When the input image is a color image, the gray conversion unit 211 converts, for example, 24-bit image data into, for example, 8-bit image data. The binarization unit 212 converts the 8-bit image data gray-converted by the gray conversion unit 211 into pixels belonging to the foreground, such as characters, lines, pictures, and photographs, included in the input image, with a logical value “1”. ”ON pixels (for example, black pixels), pixels that belong to the background area are set to OFF pixels (for example, white pixels) whose pixel value is logic“ 0 ”, and each pixel value is binarized into 1-bit binary image data To do.

  The contour extraction unit 213 extracts pixels indicating the contour of the region composed of ON pixels from the binary image data binarized by the binarization unit 212, and extracts the contour of the region composed of ON pixels. Contour binary image data to be shown is generated.

  Specifically, for example, as illustrated in FIG. 5, when the target pixel is “X” and the surrounding eight pixels are “A” to “H”, the contour extracting unit 213 determines that the target pixel “X” is an OFF pixel. In this case, it is determined that the target pixel “X” is not a pixel indicating the outline of the region formed by the ON pixels. Even when the target pixel “X” is an ON pixel and all of the surrounding eight pixels “A” to “H” are ON pixels, the target pixel “X” indicates the contour of the region composed of the ON pixels. Judge that it is not a pixel. On the other hand, when the pixel of interest “X” is an ON pixel and any of the surrounding 8 pixels “A” to “H” is an OFF pixel, the pixel of interest is a pixel indicating the outline of a region composed of ON pixels. Judge.

  The reduction unit 214 performs a process of reducing the image to 1/2, for example, with respect to the contour binary image data extracted by the contour extraction unit 213. The reason why the reduction process is performed on the input image is to increase the processing speed when the skew angle is roughly detected in the next-stage approximate angle detection unit 215. In addition, when the skew angle is roughly detected, there is an advantage that the detection accuracy can be improved by reducing the input image.

  The approximate angle detection unit 215 is configured to roughly detect the skew angle of the input image based on the contour binary image data detected by the coordinate extraction unit 2151, the Hough transform unit 2152, and the angle calculation unit 2153 and subjected to the reduction process by the reduction unit 214. Specifically, for example, detection is performed in units of several degrees.

  In the approximate angle detection unit 215, the coordinate extraction unit 2151 extracts the coordinates of each ON pixel of the reduced contour image based on the contour binary image data after the reduction process. The Hough transform unit 2152 performs a well-known Hough transform process on the contour pixel group whose coordinates have been extracted by the coordinate extraction unit 2151 to convert (vote) it into polar coordinates (ρ−θ). The angle calculation unit 2153 calculates the inclination angle of the figure formed by the outline pixel group as the approximate skew angle of the input image based on the result of the Hough conversion by the Hough conversion unit 2152. Specifically, the angle calculation unit 2153 calculates the inclination angle of the straight line in the xy plane as the skew angle of the input image based on the Hough conversion result in the Hough conversion unit 2152.

  The frequency multipoint extraction unit 216 extracts one or more points (points) having a high frequency (number of votes) in the polar coordinate (ρ−θ) space obtained by the Hough transform in the Hough transform unit 2152. The inverse Hough transform unit 217 returns the points in the polar coordinate space to the xy plane by performing an inverse Hough transform process on one or more frequently extracted points extracted by the frequent multipoint extraction unit 216. If a point in the polar coordinate space is returned to the xy plane, a straight line is obtained.

  That is, by extracting a point having a high frequency in the polar coordinate space, and applying the inverse Hough transform to the extracted point and returning it to the xy plane, it is useful for skew angle detection (suitable for skew angle detection). ) Pixel group candidates can be extracted. Here, the pixel group candidate useful for skew angle detection is a pixel group having a high linearity and a large number of pixels, specifically, a pixel group (a set of pixels) that forms a figure such as a long straight line or an elongated rectangle. That is.

  Here, one or more points (ρ, θ) with a high frequency (the number of votes) are extracted in the polar coordinate space, and the extracted points are subjected to inverse Hough transform and returned to the xy plane. Thus, a pixel group useful for scan angle detection, such as a straight line, is obtained, but instead of using an inverse Hough transform process, one or more frequent points (ρ, θ) are used on the xy plane. It is also possible to obtain a straight line.

  In order to return the binary image data subjected to the inverse Hough transform by the inverse Hough transform unit 217 to the original image size, that is, the image size before being reduced to ½ by the reduction unit 214, the enlargement unit 218 Performs the process of enlarging to double.

  The expansion unit 219 performs expansion processing on the ON pixels of the image based on the binary image data expanded by the expansion unit 218. Specifically, for example, as illustrated in FIG. 5, when the target pixel is “X” and the surrounding eight pixels are “A” to “H”, the expansion unit 219 determines that the target pixel “X” and its surroundings If there is at least one ON pixel among the eight pixels “A” to “H”, the target pixel “X” is set as the ON pixel, and all of the target pixel “X” and the surrounding eight pixels “A” to “H” If is an OFF pixel, the target pixel “X” is set as an OFF pixel.

  This expansion process is not limited to a mode in which the expansion process is performed by focusing attention on “A” to “H” for the pixel of interest “X” and the surrounding eight pixels. For example, the pixel of interest “X” Of course, the dilation processing may be performed by paying attention to a 5 × 5 pixel area centering on the pixel or a larger area.

  The logical product unit 220 performs a logical product of the binary image data extracted by the contour extraction unit 213 and the binary image data expanded by the expansion unit 219. In this way, by taking the logical product of the binary image data after the contour extraction and the binary image data after the expansion process, the slope of the straight line caused by the calculation error in the calculation process such as the Hough transform or the inverse Hough transform The error can be eliminated, and it can be taken out as binary image data of a straight line on the xy plane having a straight line inclination on the original image.

  The frequency multipoint extraction unit 216, the inverse Hough transform unit 217, the enlargement unit 218, the expansion unit 219, and the logical product unit 220 correspond to useful pixel group extraction means in the claims, and the Hough transform in the approximate angle detection unit 215. The pixel group on the xy plane is extracted on the basis of one or more points having a high frequency in the polar coordinate space by processing, and the detailed angle detection unit 223 is passed to the pixel group.

  The labeling unit 221 executes a labeling process for assigning the same label to the continuous ON pixels in the image based on the binary image data for the binary image data output from the logical product unit 220. The predetermined feature label extracting unit 222 selects a continuous pixel group whose feature amount satisfies a predetermined standard in the continuous pixel group to which the same label is given by the labeling process in the labeling unit 221, and the detailed skew angle in the detailed angle detection unit 223. For example, a continuous pixel group representing a long straight line.

  In this way, the detailed skew angle is detected by performing the labeling process on the binary image data output from the AND unit 220 and extracting the continuous pixel group whose feature amount satisfies the predetermined standard. Only useful linear components can be extracted, and other dust (noise) components can be removed.

  However, the labeling unit 221 and the predetermined feature label extraction unit 222 are not necessarily necessary components, and the logical product unit 220 can output only binary image data corresponding to a linear component useful for detecting a detailed skew angle. If present, the labeling unit 221 and the predetermined feature label extraction unit 222 may be omitted.

  The detailed angle detection unit 223 includes a coordinate extraction unit 2231, a Hough conversion unit 2232, and an angle calculation unit 2233, and is output from the logical product unit 220 and input via the labeling unit 221 and the predetermined feature label extraction unit 222. Based on the binary image data, the skew angle of the input image is detected in detail, specifically, in a detection unit smaller than the detection unit in the approximate angle detection unit 215.

  In the detailed angle detection unit 223, the coordinate extraction unit 2231 extracts the coordinates of each ON pixel of a straight line based on the binary image data input from the logical product unit 220 via the labeling unit 221 and the labeling unit 221. The Hough transform unit 2232 performs a Hough transform process on the pixel group extracted by the coordinate extraction unit 2231 based on the approximate skew angle detected by the approximate angle detection unit 215. Based on the result of the Hough transform by the Hough transform unit 2232, the angle calculation unit 2233 uses the inclination angle of the straight line as the detailed skew angle of the input image, and is a unit more detailed than the unit for calculating the approximate skew angle (for example, 0.1 degree). Unit). This detailed skew angle is the final skew angle of the input image.

  Based on the detailed skew angle detected by the detailed angle detection unit 223, that is, the skew angle of the input image, the skew correction unit 224 performs skew correction by rotating the input image so that the skew angle becomes zero. As a method for rotating the image, for example, a known method such as affine transformation can be used as in the case of the first embodiment.

  In the image processing apparatus 20B according to the second embodiment having the above-described configuration, the approximate angle detection unit 215 detects the approximate skew angle, and the detailed angle detection unit 223 detects the detailed skew angle based on the approximate skew angle. The skew angle of the input image is detected by the angle detection process in two steps by the approximate angle detection unit 215 and the detailed angle detection unit 223. However, the skew angle of the input image is not limited to the angle detection process in two steps, and is more than three steps. Multi-stage angle detection processing is also possible.

  In this case, a pixel group on the xy plane is based on one or more points in the polar coordinate space by the Hough transform processing in the previous angle detection unit between the multi-stage angle detection units. For example, a useful pixel group extraction unit including a frequent multipoint extraction unit 216 and an inverse Hough transform unit 217 may be provided.

  The components of the image processing apparatus 20B having the above-described configuration, that is, the gray conversion unit 211, the binarization unit 212, the contour extraction unit 213, the reduction unit 214, the approximate angle detection unit 215, the frequency multipoint extraction unit 216, and the inverse Hough conversion unit 217 The enlargement unit 218, the expansion unit 219, the logical product unit 220, the labeling unit 221, the predetermined feature label extraction unit 222, the detailed angle detection unit 223, and the skew correction unit 224 execute information storage processing and image processing by executing predetermined programs. It may be realized by a software configuration using a computer device that executes functions such as processing and arithmetic processing.

  However, the present invention is not limited to the implementation by software configuration, and can also be implemented by a hardware configuration or a combined configuration of hardware and software. When realized by a software configuration, the gray conversion unit 211, the binarization unit 212, the contour extraction unit 213, the reduction unit 214, the approximate angle detection unit 215, the frequency multipoint extraction unit 216, the inverse Hough conversion unit 217, the enlargement unit 218, A program that causes a computer to function as the expansion unit 219, the logical product unit 220, the labeling unit 221, the predetermined feature label extraction unit 222, the detailed angle detection unit 223, and the skew correction unit 224 is an image processing program according to the present invention.

  It can also be said that the program for executing the processing of each step in the image processing sequence described below is the image processing program according to the present invention. These image processing programs may be installed in the computer in advance. However, it may be provided by being stored in a computer-readable storage medium instead of being installed in advance, or distributed via wired or wireless communication means. .

(Skew angle detection / correction processing)
Next, an example of a procedure of skew angle detection / correction processing according to the second embodiment in which a skew angle of an input image is detected and skew correction is performed based on the detected angle will be described with reference to the flowchart of FIG.

  First, binary image data obtained by binarizing multi-valued image data or directly input binary image data is captured (step S21). Next, the captured binary image data is composed of ON pixels. The pixel indicating the contour of the region to be extracted is extracted, contour binary image data indicating the contour of the region composed of ON pixels is generated (step S22), and then an image is extracted from the contour binary image data. For example, a process of reducing to 1/2 is performed (step S23).

  Next, the rough skew angle of the input image is converted into polar coordinates (ρ−θ) by performing a Hough transform process on the contour pixel group based on the contour binary image data after the reduction process, and the result of the Hough transform is Based on, for example, several degrees as a unit (step S24), one or more frequent points in the polar coordinate space obtained by the Hough transform are extracted (step S25), and then the extracted frequent frequency By performing an inverse Hough transform process on one or more points, the points in the polar coordinate space are returned to the xy plane (step S26).

  Next, in order to restore the original image size to the binary image data subjected to inverse Hough transform, a process of enlarging the image twice is performed (step S27), and then the image based on the enlarged binary image data is processed. Expansion processing is performed on the ON pixels (step S28), and then logical product of the binary image data after the expansion processing and the binary image data obtained by contour extraction from the binary image data is obtained (step S29). ).

  Next, a labeling process is performed on the binary image data after the AND operation to give the same label to consecutive ON pixels in the image based on the binary image data (step S30), and then the same In the continuous pixel group to which a label is given, a continuous pixel group whose feature amount satisfies a predetermined criterion is extracted as a continuous pixel group used for detecting a detailed skew angle (step S31), and then Hough transform is performed based on the approximate skew angle. The detailed skew angle is detected as the final skew angle of the input image using the technique (step S32). Then, the skew correction is performed by rotating the input image based on the calculated detailed skew angle (step S33), and the series of skew angle detection / correction processing ends.

  As described above, after performing the Hough transform process on the binary image data extracted from the contour, useful pixels on the xy plane based on one or more points having a high frequency in the polar coordinate space by the Hough transform process. By extracting a group, a pixel group that forms a figure such as a long straight line, a long and narrow rectangle, or a large rectangle can be extracted as the useful pixel group. Then, by performing angle detection by Hough transform on the extracted pixel group and performing skew angle detection, the pixel group that forms the most suitable, simplest, and most accurate figure for skew angle detection is obtained. Since the Hough conversion process is performed only in the above-described manner, the accuracy of detecting the skew angle can be increased.

  Also, instead of performing the Hough transform process on the entire input image, only the pixel group useful for skew angle detection is extracted from the second stage onward for the binary image data extracted at the first stage angle detection unit. By performing the Hough transform process, characters that do not contribute to skew angle detection, short straight lines, small rectangles, and other figures can be excluded from the computation process. The number of target pixels can be minimized. As a result, it is possible to reduce the time required for the arithmetic processing in the Hough transform, so that it is possible to improve the processing speed of a series of processes for detecting the skew angle and correcting the skew based on the skew angle.

It is a block diagram which shows the structural example of the image processing system containing the image processing apparatus by this invention. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 1st Embodiment of this invention. It is a flowchart which shows an example of the procedure of the skew angle detection and correction process which concerns on 1st Embodiment. It is a block diagram which shows the function structural example of the image processing apparatus which concerns on 2nd Embodiment of this invention. It is explanatory drawing of the contour extraction by a contour extraction part. It is a flowchart which shows an example of the procedure of the skew angle detection and correction process which concerns on 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Image data input part, 20, 20A, 20B ... Image processing apparatus (image processing part), 30 ... Image data output part, 21 ... CPU, 22 ... I / O circuit, 23 ... ROM, 24 ... RAM, 25 ... HDD (Hard Disk Drive) device, 26 ... bus line, 201, 212 ... binarization unit, 202, 221 ... labeling unit, 203 ... specific pixel group extraction unit, 204, 2151, 2231 ... coordinate extraction unit, 205, 2152, 2232 ... Hough transform unit, 206, 2153, 2233 ... Angle calculation unit, 207, 224 ... Skew correction unit, 214 ... Reduction unit, 215 ... Rough angle detection unit, 216 ... Frequency multi-point extraction unit, 217 ... Inverse Hough transform unit 218 ... Enlargement unit, 219 ... Expansion unit, 220 ... Logical product unit, 222 ... Predetermined feature label extraction unit, 223 ... Detailed angle detection unit

Claims (4)

Contour extracting means for extracting pixels indicating the contour of the binary image obtained by binarizing the input image;
Multi-stage angle detection means for detecting the inclination angle of the input image in stages by Hough transform processing on the binary image data extracted by the contour extraction means;
A useful pixel group on the xy plane is extracted based on one or more points having a high frequency in the polar coordinate space by the Hough transform process in the previous angle detecting means between the multi-stage angle detecting means. Useful pixel group extraction means to be passed to the subsequent angle detection means,
An image processing apparatus comprising: a skew correction unit configured to perform skew correction on the input image based on an angle detected by the last-stage angle detection unit in the multi-stage angle detection unit. The useful pixel group extracting means includes
A frequency multipoint extraction means for extracting one or more points having a high frequency in the polar coordinate space;
The image processing apparatus according to claim 1, characterized in that it has an inverse Hough transform means for performing inverse Hough transform processing to the one or more points often frequently extracted by the frequency multipoint extracting means. Expansion means for performing expansion processing on each pixel of the useful pixel group extracted by the useful pixel group extraction means;
A logical product unit that takes a logical product of the binary image data expanded by the expansion unit and the binary image data extracted by the contour extraction unit and passes the logical product to the subsequent angle detection unit. The image processing apparatus according to claim 1, wherein: A contour extracting step for extracting pixels indicating the contour of the binary image obtained by binarizing the input image;
A multi-step angle detection step of detecting the tilt angle of the input image stepwise by a Hough transform process on the binary image data extracted in the contour extraction step;
A useful pixel group on the xy plane is extracted based on one or more points having a high frequency in the polar coordinate space by the Hough transform process in the angle detection step in the previous stage between the multi-stage angle detection steps. Useful pixel group extraction step passed to the angle detection step at the later stage,
An image processing program that causes a computer to execute each processing of a skew correction step of performing skew correction on the input image based on an angle detected in a final angle detection step in the multi-stage angle detection step.

Images