JP2010250387A - Image recognition device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a process for detecting a space area, with high speed, from a document image having color and gray gradation acquired by scanning a paper document. <P>SOLUTION: In an image which is obtained by scanning the paper document having color and gray gradation, the profile shape of the space area is calculated, by calculating the boundary threshold between a foreground color and a background color by distribution analysis of image density, even when a dark color is designed at near edge of the space. An image edge is extracted at a high speed, by performing binary search into each directions perpendicular to each side close to the boundary side of the acquired space area, then four sides detection in which the calculation amount is related less with respect to a high-resolution color and gray gradation image is achieved, by minutely calculating the lines that constitute four sides. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image recognition apparatus and a program, for example, to a technique of capturing a paper document by an OCR apparatus or the like, and cutting out and storing only a portion that seems to be a document area from the captured image.

  Due to social demands such as legal compliance and document digitization, there is a social demand for document reading systems that store various business documents as electronic images and read them. In general, an OCR (Optical Character Recognition) apparatus is used to perform such a process. That is, a paper document is scanned using an OCR device, an area of the paper document is detected from the image, characters written in the paper document area are recognized, and the reading result is displayed on the correction interface. Various business documents are stored as electronic images by a series of processes in which detection errors, reading errors, and reading defects in the paper area existing in the reading result are manually corrected.

  At this time, one of the important issues for the OCR apparatus and the document reading system is to store a correct document image while reducing the capacity as much as possible for documents having various designs and colors. For this purpose, it is necessary to estimate the area of the document from the image, extract an image of only the area, further correct (rotate correction, cut out, etc.), and save this. Here, the process of estimating an area that seems to be a paper region is referred to as four-side detection in the sense that four sides (edges) of the paper surface are detected.

JP-A-8-162190 JP 2005-285010 A

  Although there has been a conventional technique for detecting four sides of a binary image, detection of four sides for a document having various designs, patterns, and colors has not yet been established. In the conventional method, a binary image is used, or a color image is scanned to extract a portion that seems to be an edge by a filter operation or the like, and a four-side detection is performed by tracing the outline of a paper area (for example, Patent Document 1). That is, the conventional OCR scanner performs four-side detection using a binary image on the assumption that the paper area (foreground color) is white and the background color is black. In this case, four sides are detected by creating a run from a binary image, tracking a white / black boundary line on the run, and calculating a contour surrounding the paper area. Therefore, when it is assumed that a binary image is input, there is a problem in that it cannot be detected accurately when a dark color is placed up to the edge of the paper. As a countermeasure, it is conceivable to perform four-side detection using a color image or a gray gradation image.

  However, since the capacity of a color image is greatly increased to 4 to 24 times that of a binary image, there is a problem that the processing time greatly increases in the method that follows the binary image.

  In addition, black, which is the background color of the paper surface, does not have a stable luminance due to the influence of paper dust attached to the OCR device, and there is a problem that an estimation error of the paper surface region is likely to occur.

  Furthermore, as described above, scanning the entire surface of a high-resolution color image or gray gradation image requires a calculation amount, and thus there remains a problem in terms of processing time.

  The present invention has been made in view of such a situation, and provides a technique capable of accurately and rapidly detecting four sides even when a high-resolution color image is a target.

  In order to solve the above-described problem, in the present invention, a rough paper area is estimated by pre-sampling from an image in a lattice point shape and estimating the boundary between the background color and the foreground color by analyzing the distribution of the image constituent colors. . Then, by performing a detailed binary search in the vicinity of the rough region, the amount of calculation is suppressed and the edge is detected. Also, a test process for validity of the edge side is performed, and if necessary, more detailed edge side detection is performed. Thus, in the present invention, edge detection processing with a multi-stage configuration is executed.

  More specifically, an image recognition apparatus (OCR apparatus) according to the present invention includes a lattice point extraction processing unit, an image analysis unit, an edge estimation processing unit, and a boundary line acquisition unit. The lattice point extraction processing unit extracts a plurality of lattice points from the image data. The image analysis unit generates a first histogram relating to the image density using the extracted plurality of grid points, calculates a first separation threshold for separating the background and the foreground from the first histogram, The first separation threshold is applied to a plurality of grid points to extract grid points near the boundary between the background and the foreground. The edge estimation processing unit also performs edge estimation using lattice points near the boundary. This estimation can be realized by executing a binary search process on a plurality of grid points belonging to the foreground and background closest to the first separation threshold. The boundary line acquisition unit acquires a boundary line by executing a straight line approximation process using a plurality of edge points obtained by the edge estimation process.

  The image recognition apparatus further includes a boundary evaluation unit. The boundary evaluation unit evaluates whether the boundary line is appropriate as a boundary for distinguishing between the foreground and the background, using a difference value between the image density in the foreground region and the image density in the background region. Specifically, the boundary evaluation unit determines whether or not the number of locations where the difference value of the image density is equal to or less than the first threshold is less than a predetermined value (first condition) and the variance of the image density in the background region. It is determined whether or not it is larger than the second threshold (second condition), and if either the first or second condition is satisfied, it is determined that the acquired boundary line is valid. If both the first and second conditions are not met, the boundary line is determined to be inappropriate. In this case, the image analysis unit acquires a predetermined number of pixel samples existing in the vicinity of the acquired boundary line, generates a second histogram for the pixel sample, and generates the second histogram from the second histogram. A second separation threshold for separating the foreground is calculated. Then, the edge estimation processing unit detects a plurality of edge points (corrected edge points) by executing a binary search process using the second separation threshold. Further, the boundary line acquisition unit acquires a corrected boundary line by executing a straight line approximation process using the corrected edge point. In this way, multistage aggressive edge detection processing is realized.

  The image recognition apparatus further includes an inclination correction unit. When there are a plurality of boundary lines or correction boundary lines, the inclination correction unit calculates the inclination difference between two opposing boundary lines or correction boundary lines, and when the inclination difference is equal to or greater than a predetermined value, Correct the corrected boundary line.

  Further features of the present invention will become apparent from the best mode for carrying out the present invention and the accompanying drawings.

  According to the present invention, for an image obtained by scanning a paper document with color or gray gradation, even if a dark color is placed on the edge of the paper, the separation boundary between the background color and the foreground color is estimated by the distribution analysis of the image constituent color, This can be separated. Further, by performing a detailed boundary search by binary search, high-speed four-side detection can be expected even for a high-resolution image.

It is a figure which shows an apparatus structure. It is a conceptual diagram of 4 side detection. It is a functional block diagram of 4 side detection processing. It is an example of the image density distribution analysis in 4 side detection. It is an example of determination of area estimation and binary search area in 4 side detection. It is an example of the image density distribution analysis in the second pass in the four-side detection. This is an example of correcting one side with low likelihood in four-side detection. It is an example of an image assuming processing in the second pass.

  The present invention relates to, for example, a technique for capturing a paper document with an OCR device or the like, and cutting out and storing only a portion that is considered to be a document area from the captured image. In order to realize this, in the present invention, pre-sampling is performed from an image in a lattice point shape, a rough paper area is estimated by estimating the boundary between the background color and the foreground color by analyzing the distribution of image constituent colors, and By performing a detailed binary search in the vicinity of a rough region, the amount of calculation is suppressed and edge detection is performed.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be noted that this embodiment is merely an example for realizing the present invention, and does not limit the technical scope of the present invention. In each drawing, the same reference numerals are assigned to common components.

<Configuration of recognition device (OCR device)>
First, a hardware configuration to which the present embodiment is applied will be described. FIG. 1 is a diagram showing a schematic configuration of a character recognition device (an OCR device and a document reading system) according to an embodiment of the present invention.

  In the OCR device 0100, a paper document is converted into electronic data by an image capturing device 0101 that is an image capturing unit, and the electronic document is stored in an external storage device 0105 and a memory 0106 that are storage units, and is a central processing unit (CPU). Reading is performed by the arithmetic unit 0107.

  The OCR program and the trail management program according to the present embodiment are stored in the external storage device 0105 or the memory 0106 or introduced into the device via the communication device 0109 and stored in the storage unit 0105 or 0106. In the OCR program, the central processing unit 0107 performs image processing on the captured electronic data image, performs four-side detection, and outputs an image obtained by cutting out only the paper area if necessary. These processing results can be operated (corrected) by a human operator through the operation terminal device 0102, and the processing results and the correction results are displayed on the display terminal device 0103. Information such as processing results may be stored in the external storage device 0105 or transmitted as data to the external connection device through the communication device 0109 as necessary.

  The processing units and devices described above are connected by an internal bus 0108. The input slips are distributed and collected in a box defined by the sorter device 0104 for each size and type of slip. In other words, the OCR device 0100 can be configured by a computer system such as a normal personal computer (PC) except for the image capturing device 0101 and the sorter device 0104.

  The OCR program displays the image output from the OCR device 0100 and the recognition result on the display terminal device 0103. The detection of the four sides of the image is corrected and checked by the person who is the operator through the operation terminal device 0102. Then, the OCR program stores the correction result in the external storage device 0105 or the memory 0106.

<Specific processing contents of four-side detection>
FIG. 2 is a diagram illustrating the concept of four-side detection. FIG. 3 is a flowchart for explaining an outline of the four-side detection process. Further, FIGS. 4 to 8 show examples of four-side detection processing and images, and are diagrams for helping understanding of the contents of the four-side detection processing.

  Hereinafter, each processing step will be described in detail with reference to FIGS. The operating subject of each step is the central processing unit 0107 unless otherwise specified. The central processing unit 0107 functions as each processing unit when processing each step. That is, for example, when the pre-sampling process is operated, the central processing unit 0107 serves as a pre-sampling processing unit. The same applies to the other steps.

1) Pre-sampling process (S0301)
In this step, an Ln × Ln grid is fitted to the input image (for example, a form image), and pixel information of the grid point is sampled. That is, this is a process of defining a plurality of grid points for the input image and estimating the color (density) of the pixels near the grid points. It is sampled not only on paper but also on a black background.

  The number of Ln is determined from the minimum paper surface size to be detected and the maximum area that can be imaged. For example, if it is desired to include a minimum of 4 × 4 sampling points on the minimum paper size, the number of grid points is determined in the form of (4 × horizontal width of the maximum area size) / horizontal width of the minimum paper size. Although it can be increased to improve accuracy, the processing time increases accordingly.

  In this step, image sampling processing is further executed for each lattice point. At this time, the median value of Pn × Pn pixels is used as the pixel value. When the median value of Pn × Pn pixels is used, the noise component does not affect the median value even if (Pn / 2-1) pixel-wide line-shaped noise crosses the target region.

  Therefore, a value twice as large as the assumed noise width in the scanning direction at the time of scanning may be set as Pn. In general, the number of pixels corresponding to 1 mm as Pn, that is, Pn = 8 is sufficient for a 200 dpi image.

2) Threshold determination process by image density distribution analysis (S0302)
In this step, a histogram of image constituent colors is generated from Ln × Ln sample points obtained by pre-sampling.

  FIG. 4 shows an example of the histogram. In the histogram 0401 illustrated in FIG. 4, the data section 50 is background color data, and a first peak is formed. Therefore, it can be seen that the background color and the form color can be identified by setting the threshold value Tb1 to the value 100 next to this peak. Specifically, the horizontal axis of the histogram of image density distribution is scanned in ascending order from 0 (black), the first peak is found, and the background color and foreground color are lower than the Tp ratio of the number of samples in the first peak. The boundary of In general, 0.1 is preferably used as Tp.

3) Edge estimation process (S0303)
The edge estimation process executed in this step is composed of two stages.

  First, using the information of Ln × Ln lattice points obtained by pre-sampling, the process of estimating the position of the edge (four sides) of the form is executed. This is to calculate which block (small region surrounded by the grid) the edge is by operator calculation for the pixel value on the grid point (the median value of the Pn × Pn pixels). Specifically, two types of pseudo Sobel operators in the vertical direction and the horizontal direction are used in order to identify four boundary lines in the vertical and horizontal directions. FIG. 5A is a diagram illustrating an example of sampling and edge estimation intervals. Sampling points are indicated by circles, estimated regions are indicated by gray circles (0501), and sections for further detailed determination are indicated by bold lines (0501).

  Next, the edge is calculated in detail by a binary search in the small region obtained by the edge estimation. The binary search is a process of performing edge estimation on the four sides from the end of the bitmap and detecting the edge by pinching the edge from both ends. Since the pixel values at both ends and the center thereof are compared in the binary search, edge detection can be performed even in the case of a dark ground color form. That is, if the pixel values at both end points are G1 and G2, and the intermediate pixel value is G3, not only the absolute value of the difference amount | G1-G2 | This is because it is possible to make a judgment based on the amount of change. This will be described with reference to the example of FIG. 5B.

  FIG. 5B is a diagram illustrating changes in pixel values in a certain range in which a binary search is performed. The pixel value (0503) of the point first determined to be within the boundary (on the paper surface) exceeds the threshold Tb1, but it is assumed that a slightly darker color is applied with gradation on the paper surface. The area up to the adjacent point is the area on the paper. That is, there is a possibility that it is erroneous to determine the boundary between the paper surface (bright region) and the background (dark region) only by the threshold value Tb1. Therefore, a search is made for a place where the pixel values are converted most steeply in a binary search G1 → G3 → G2. The both end points at the beginning of the search are (0504, 0505), the intermediate point is (0506), and the steepness of the change of the pixel value can be measured by the angle (0507). When the calculation is repeated so that the larger difference between | G1-G3 | and | G3-G2 | is used as the end point of the next binary search so as to obtain a steeper change amount, the end point is finally set as ( 0508, 0509), the middle point (0510), and the largest change amount (0511). Thereby, it can be estimated that a portion that can be estimated as an edge in the edge estimation section is between the point (0508) and the point (0510).

4) Boundary line approximation processing (S0304)
In this step, edge detection at the maximum Ne places is executed for each side, and linear approximation is executed for the obtained edge group by the least square method. In other words, this is a process for linearly approximating a point that seems to be a boundary point obtained by the edge estimation process.

  In the linear approximation, those having a large deviation from the approximated straight line are excluded from the approximation, and this is repeated several times to eliminate the influence of erroneous recognition due to corner cuts or noise. As a result of the preliminary experiment, convergence is achieved by 5 to 7 comparisons per location, and edges are detected by Ne × 7 comparisons for each side, and a maximum of 4 × Ne × 7 comparisons for the total of the four sides. Generally, 10 is used as Ne.

5) Approximate boundary evaluation process (S0305)
In this step, Ne points are sampled in the two areas separated by the approximate boundary line, and the validity of the boundary line is evaluated based on the difference value of the image density inside and outside the paper area. That is, this is a process of determining whether the obtained approximate straight line is likely to be a boundary line.

If the difference between the pixel values inside and outside the boundary line is small, it is not suitable as a boundary for dividing the region. Further, when the variance of the pixel values outside the boundary line is large, it is determined that various patterns are placed and it is determined to be inappropriate. That is, for the boundary line that does not satisfy the following condition a or b, the processing after step S0306 is executed. If the following condition is satisfied, the approximate boundary line is determined to be appropriate, and the process proceeds to step S0309.
a) For the sampling position along the boundary line, the difference amount of the pixel value is calculated between the inside and the outside of the boundary line, and the number of locations where the difference amount is equal to or less than the threshold value Te1 is less than the threshold value Te2 (number).
Or b) It is determined that the dispersion of the image density outside the paper area is larger than the threshold Te3 (determining how much the color of the area determined to be outside the paper varies).

  For a boundary line that meets either of these two conditions, the boundary line is corrected by a second pass process for the boundary line. The processing of the second pass will be described in the following 6) to 8). The second pass process is a process for obtaining an approximate boundary straight line by analyzing the image density more partially (finely) than in S0302.

6) Image density distribution analysis process 2 (S0306)
This step is the first process of the second pass, and here, the threshold value is determined again by image density distribution analysis.

  First, No pixel samples are obtained for each boundary line for pixel samples outside the boundary and pixel samples just outside the boundary line in order to detect borders, and a histogram of these pixel samples is created.

  FIG. 6 is a diagram showing an example of this histogram. The hatched area in FIG. 6A becomes the sampling area, and the histogram obtained therefrom is as shown in FIG. 6B.

  In the histogram of FIG. 6B, it can be seen that the left side of the horizontal axis is a dark part (background), the right side is a bright part, and a valley is formed between them. This valley is defined as a threshold value Tb2 between the background color and the foreground color. More specifically, the threshold value is calculated by scanning the data section in ascending order from 0 and setting the section where the frequency has decreased to the Tp ratio from the first peak. The difference between the threshold value Tb1 and the threshold value Tb2 is the size of an area used when creating a histogram. Since the histogram from which the threshold value Tb2 is calculated is limited to the region estimated as the initial boundary line, the boundary between the background color and the foreground color is more likely to appear.

7) Edge detection process (S0307)
In this step, an edge is detected by the binary search in the process of S0303. Note that the threshold used at this time is the threshold obtained in the process of S0306.

8) Boundary line approximation processing (S0308)
In this step, the boundary line is approximated by a straight line from the edge point obtained in S0307. Regarding the boundary line approximation, the same processing as that executed in S0304 is executed.

9) Parallel line parallelism evaluation process (S0309)
In this step, the inclinations of the two opposing boundary lines are compared, and if the difference between the inclinations is greater than or equal to a certain value, the boundary line correction processing is executed in S0310. That is, when the directions of two boundary lines on the left and right, or on the upper and lower sides are expressed as vectors (two-dimensional vectors) L1 and L2, the absolute cosine value of the angle between the vectors | <L1, L2> | / (| L1 | If x | L2 |) is larger than the threshold value Tr, it is determined that both are not parallel lines.

10) Boundary line correction processing (S0310)
In this step, for the two boundary lines determined not to form parallel lines in the process of S0309, the angle at which each boundary line intersects with the vertically adjacent boundary line is obtained. If there is an error Ta above a certain level from the perpendicular to the angle, the boundary line is considered to be wrong, and the boundary line is estimated from the opposing boundary line. As shown in FIG. 7, such processing is performed when a pattern having the same darkness as the background color is placed on the paper surface and occupies a large area. This is to help. In this case, the straight line (0701) was initially estimated as the rightmost boundary line. However, since the left, upper, and lower boundary lines have a high likelihood, these three sides are regarded as accurate, and the ambiguous right side is obtained again. The process consists of the following steps.

i) Process of obtaining a sample point farthest from the left side and having a density equal to or higher than the foreground threshold.
ii) Process in which the right side is a line segment having the same inclination as the left side and passing through the farthest sample point. By the above processing, a new boundary line 0702 is obtained.

  On the other hand, if two boundary lines determined not to constitute a parallel line in the processing of S0308 both have an angle error Ta or more, the angle of the other boundary line is estimated based on what is more likely to be a boundary line. . In this case, more likely to be a boundary line, as in the condition a of S0305, select sampling points along the boundary line, take the amount of difference between the pixel values inside and outside the boundary, Judge that it seems to be a boundary line.

  The above is the process of detecting four sides. The difference between the roles of the first pass and the second pass can be summarized as follows.

  R1) In the first pass, a recognition process is executed with a threshold value determined from set values in all data that have been sampled in advance.

  R2) In the second pass, using the statistical information collected in the first pass, the threshold value is determined from the data outside the area determined by changing the internal set value, and the recognition process is executed.

  In the preliminary experiment, for a document image having various designs, the first pass finds an accurate side (edge) for 90% of the image samples. Reestimation is performed in the second pass because the remaining 10% of the form has a dark border. For the latter, by using the image density distribution information, the side can be correctly re-estimated by calculating a threshold value for identifying the background color and the foreground color.

  As an image that is assumed to be processed in the second pass, for example, as shown in FIG. 8, the edge of the paper surface is not straight, but is distorted or shredded, or a dark color is designed at the paper edge. There are things, etc.

<Summary of Embodiment>
As described above, in the embodiment of the present invention, a histogram is created for a plurality of grid points obtained by sampling an input image, an approximate edge existence region is estimated based on the distribution state, and the region is stored in the region. On the other hand, a binary search process is performed. As a result, for a scanned image of a paper document in color or gray scale, even if a dark color appears on the edge of the paper, the separation boundary between the background color and the foreground color is estimated by analyzing the distribution of the image composition color. Can be separated.

  If the boundary line approximated from the edge obtained by the first binary search process is not valid, a histogram is further generated for the pixels near the obtained boundary line. A threshold value for separating the foreground and the background is obtained from the histogram, and a more detailed boundary search is performed by the binary search process (second time) using the threshold value. This enables high-speed four-side detection even for high-resolution images.

  The present invention can also be realized by a program code of software that realizes the functions of the embodiments. In this case, a storage medium in which the program code is recorded is provided to the system or apparatus, and the computer (or CPU or MPU) of the system or apparatus reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the program code itself and the storage medium storing the program code constitute the present invention. As a storage medium for supplying such program code, for example, a flexible disk, CD-ROM, DVD-ROM, hard disk, optical disk, magneto-optical disk, CD-R, magnetic tape, nonvolatile memory card, ROM Etc. are used.

  Also, based on the instruction of the program code, an OS (operating system) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing. May be. Further, after the program code read from the storage medium is written in the memory on the computer, the computer CPU or the like performs part or all of the actual processing based on the instruction of the program code. Thus, the functions of the above-described embodiments may be realized.

  Also, by distributing the program code of the software that realizes the functions of the embodiment via a network, the program code is stored in a storage means such as a hard disk or memory of a system or apparatus, or a storage medium such as a CD-RW or CD-R And the computer of the system or apparatus (or CPU or MPU) may read and execute the program code stored in the storage means or the storage medium when used.

0100: OCR device 0101 ... Image pickup device 0102 ... Operation terminal device 0103 ... Display terminal device 0104 ... Sorter device 0105 ... External storage device 0106 ... Memory 0107 ... Center Arithmetic device 0108 ... Internal bus 0109 ... Communication device

Claims (7)

An image recognition apparatus that recognizes image data obtained by scanning and having a background and a foreground,
A grid point extraction processing unit for extracting a plurality of grid points from the image data;
Generating a first histogram relating to image density using the plurality of extracted grid points, calculating a first separation threshold for separating the background and the foreground from the first histogram; An image analysis unit that applies the first separation threshold to the grid points and extracts grid points in the vicinity of the boundary between the background and the foreground;
An edge estimation processing unit for edge estimation using lattice points in the vicinity of the boundary;
A boundary line acquisition unit that acquires a boundary line by performing a straight line approximation process using a plurality of edge points obtained by the edge estimation processing unit;
An image recognition apparatus comprising:   The edge estimation processing unit detects edges by performing a binary search process on a plurality of grid points belonging to the foreground and background that are closest to the first separation threshold. Image recognition device.   Further, using the difference value between the image density in the foreground region and the image density in the background region, the boundary line acquired by the boundary line acquisition unit is valid as a boundary for distinguishing the foreground and the background. The image recognition apparatus according to claim 2, further comprising a boundary evaluation unit that evaluates whether or not.   The boundary evaluation unit determines whether or not the number of locations where the difference value of the image density is equal to or less than a first threshold is less than a predetermined value (first condition) and the variance of the image density in the background region is second. It is determined whether or not the second boundary is greater than the threshold value (second condition), and if the first or second condition is satisfied, the acquired boundary line is determined to be valid. The image recognition apparatus according to claim 3. The boundary evaluation unit determines that the acquired boundary line is inappropriate if it falls outside either of the first and second conditions,
In this case, the image analysis unit acquires a predetermined number of pixel samples existing in the vicinity of the acquired boundary line, generates a second histogram for the pixel sample, and generates the second histogram from the second histogram. And calculating a second separation threshold for separating the foreground,
The edge estimation processing unit detects the plurality of edge points (corrected edge points) by executing the binary search process using the second separation threshold,
The image recognition apparatus according to claim 4, wherein the boundary line acquisition unit acquires a corrected boundary line by executing a straight line approximation process using the corrected edge point.   Further, when there are a plurality of the boundary lines or the correction boundary lines, the inclination difference between two opposing boundary lines or the correction boundary lines is calculated, and when the inclination difference is equal to or larger than a predetermined value, the opposing boundary line or the correction boundary line is corrected. The image recognition apparatus according to claim 5, further comprising an inclination correction unit that corrects the boundary line.   A program for causing a computer to function as the image recognition apparatus according to claim 1.

Images