JP6272219B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to a technique for specifying a correct region from region candidates obtained from a captured image.

  With the widespread use of camera-equipped mobile phones, a technology has been developed that scans (images) various papers such as posters, whiteboards, business cards, and handouts with a camera, and converts them into electronic documents. However, a trapezoidal distortion occurs in the captured image of the paper according to the positional relationship between the camera and the paper. In order to correct such distortion and use the camera as an alternative to the conventional scanner, it is necessary to accurately acquire a paper area indicating the boundary of the paper surface in the captured image.

  In Patent Document 1, in order to detect a document boundary from an image, the inner and outer pixel values of the entire area candidate are used to determine the inner and outer separation degrees of the area candidate, and the area candidate having a high degree of separation is selected. A method is disclosed.

JP 2013-105276 A

  However, in the method of Patent Document 1, the degree of separation is obtained using pixel values inside and outside the entire region candidate. Pixel values on the inside and outside of the area candidate when the variation in pixel values is large (the variance value is high) in the area inside and outside the area candidate due to the influence of ambient light, etc., or when the background pixel value is not uniform in the first place The degree of separation may be low.

  In that case, even if it is a paper area candidate that correctly indicates the boundary of the area, if the degree of separation between the inner and outer pixel values of the whole area candidate is calculated by the method of Patent Document 1, the degree of separation becomes low. There is a problem that candidates cannot be selected correctly.

  The present invention has been made in view of such problems, and for a region imaged in an environment where the background is not uniform, the region of interest is accurately identified from the paper region candidates that correctly indicate the boundary of the region. For the purpose.

In order to solve the above-described problems, an image processing apparatus according to the present invention includes an input unit that inputs a region composed of a plurality of line segments as a region candidate, and a predetermined size or less in each line segment that forms the region candidate. In the plurality of regions, the first information indicating the relationship between the image feature amounts on both sides with the line segment as a boundary, and images on the inside and outside of the region candidate in a region having a predetermined size or less at each corner of the region candidate The region candidate is identified as a region of interest using output means for outputting second information indicating the relationship between feature amounts, and the first information and the second information corresponding to each of the plurality of regions. And a specifying means.

  According to the present invention, it is possible to accurately specify a paper area that correctly indicates the boundary of an area even for an image of an area captured in an environment where the background is not uniform.

It is a figure which shows the structure of an image processing apparatus. It is a figure which shows the outline | summary of a paper surface detection process. It is a flowchart which shows a paper surface detection process. It is a figure which shows the function structure at the time of performing an edge detection process. It is a flowchart which shows an edge detection process. It is a figure explaining 1st noise determination processing. It is a figure explaining the 2nd noise judgment processing. It is a flowchart which shows a line segment output process. It is a flowchart which shows a line segment fitting process. It is a figure which shows the concept of the isolation | separation in line segment fitting. It is a figure which shows the specific example of a line segment fitting process. It is a figure which shows the specific example of a line segment candidate selection process. It is a figure which shows the outline | summary of the addition process of an edge pixel. It is a figure which shows the function structure at the time of performing a line segment output process. It is a flowchart which shows a paper surface area | region identification process. It is a functional block diagram at the time of performing a paper surface area | region specific process. It is a figure which shows the example of a division | segmentation of the edge region of a paper surface area candidate. It is a flowchart which shows the score calculation process of a division area. It is a figure which shows the example of the evaluation value of a division area. It is a figure which shows the example of the division area of a paper surface area | region candidate. It is a figure which shows the example of the evaluation value of a division area. It is a figure which shows the specific example of the selection process of a paper surface area candidate. It is a figure which shows the example of the paper surface area | region candidate of 2nd Embodiment. It is a flowchart which shows the score calculation process of the division area in 3rd Embodiment. It is a figure which shows the example of the paper surface area | region candidate of 3rd Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, an example when the present invention is applied will be described. However, not all the features described in the present embodiment are necessarily essential to the present invention.

<First Embodiment>
Apparatus Configuration FIG. 1 is a diagram illustrating a configuration of an image processing apparatus according to the present embodiment. The image processing apparatus according to the present embodiment includes an image input unit 101 that inputs captured image data, a CPU 102, a RAM 103, and a storage unit 104. The CPU 102 executes an image processing program for performing line segment output processing of the present embodiment on image data. The RAM 103 is used for temporary storage of work memory and data when executing the program. The program and data are stored in the storage unit 104.

  The configuration of the image processing apparatus illustrated in FIG. 1 is an example, and may include components other than those illustrated here. Further, image processing may be executed using an external general-purpose computer or the like, or image processing may be executed on an electronic circuit. Furthermore, the image data input to the image input unit 101 is not limited to a captured image, and may be an artificially created image.

Outline of Paper Surface Detection Process FIG. 2 is a diagram showing an outline of the paper face detection process by the image processing apparatus of the present embodiment. FIG. 2A is an example of captured image data acquired by the image input unit 101, and a plurality of sheets of paper are arranged. When the paper surface detection process according to the present embodiment is performed on the captured image data, a quadrangle representing the area of each paper surface is obtained as shown in FIG. Hereinafter, a rectangular area formed by a straight line representing the boundary between the paper surface and the background is referred to as a paper surface region. In the present embodiment, each pixel of the captured image is described as being expressed by a luminance value. However, the present invention is not limited to image data by a luminance value.

  FIG. 3 is a flowchart showing the paper surface detection process in the present embodiment. As described above, this processing is realized by the CPU 102 executing the program.

  First, in step S301, edge detection is performed on the captured image data acquired by the image input unit 101. By this processing, pixels having a high intensity gradient in the captured image data are obtained as edge pixels. In the edge detection process, an edge corresponding to noise is removed. Here, the edge corresponding to noise is an edge that is estimated not to constitute an edge of the document.

  Next, in step S302, a straight line constituted by a sequence of edge pixels is generated from the set of edge pixels obtained in step S301. The straight line is generated by a known method such as Hough transform or Radon transform. Subsequently, in step S303, a line segment group is acquired based on the set of straight lines and edge pixels generated in step S302. In the present embodiment, a highly reliable line segment is output as a line segment indicating the boundary of the paper area from the plurality of acquired line segments. In this embodiment, a straight line has no end points, and a line segment is distinguished as having two end points.

  Subsequently, in step S304, four line segments are selected on the basis of the positional relationship of each line segment, and a quadrilateral that is a candidate for the paper area is generated. Finally, in step S305, a quadrangle indicating a paper area is specified from the quadrangle generated in step S304.

  Hereinafter, each process in the flowchart shown in FIG. 3 will be described in detail.

Edge detection processing The edge detection processing in step S301 will be described in detail with reference to FIGS. FIG. 4 is a diagram illustrating a functional configuration when performing edge detection processing in the image processing apparatus of the present embodiment, and FIG. 5 is a flowchart illustrating edge detection processing of the present embodiment.

  First, in step S501, the captured image input unit 401 inputs a processing target image. Next, in step S502, the edge image creation unit 402 creates an edge image obtained by extracting edge pixels from the input image. It is assumed that a known method such as an edge detection canny method (John F. Canny) is used to create the edge image. In step S503, the connected pixel block creation unit 403 creates a connected pixel block in which pixels are connected. Here, the connected pixel block is a pixel set in which edge pixels are connected in the vicinity of eight.

  In step S504, the circumscribed rectangle creating unit 404 creates a rectangle circumscribing each of the connected pixel blocks created in step S503. At this time, for example, in the case of the input image shown in FIG. 2A, there are both the edge of the paper surface due to the boundary between the paper surface and the background, and the edge that should be noise due to the background texture and characters in the document. Yes. When detecting a paper area from a captured image, it is desirable to remove only the edge that should be noise and extract only the edge of the paper.

  Therefore, in this embodiment, after step S505, it is determined from the various parameters indicating the characteristics of the circumscribed rectangle whether or not the connected pixel block corresponding to the circumscribed rectangle is equivalent to noise. If it is determined that the noise is equivalent, the connected pixel is removed. This is because the edge of the paper surface corresponding to the boundary region of the paper surface is generally a straight line and is longer than the edge due to noise, or the edge due to noise does not extend in a straight line.

  Hereinafter, the noise removal processing in steps S505 to S509 in the present embodiment will be described in detail.

  First, in step S505, the circumscribed rectangle parameter comparison unit 405 compares the length of the long side of the circumscribed rectangle with a predetermined threshold (first threshold). From the comparison result, the first noise determination unit 406 determines whether or not the circumscribed rectangle is a candidate for a region (hereinafter referred to as a boundary region) constituting the edge of the paper surface. This utilizes the property that a rectangle that circumscribes the edge of the paper surface has a high possibility of extending in one direction. Specifically, if the long side of the circumscribed rectangle is shorter than the first threshold value, it is determined that it is not a boundary region but noise that is not to be extracted, and the process proceeds to step S509, where the first noise removal unit 407 Remove block. On the other hand, if the length of the long side of the circumscribed rectangle is equal to or greater than the first threshold value, the process proceeds to step S506, which is the second noise determination process, while leaving this connected pixel block as a boundary region candidate.

  FIG. 6 shows a specific example of the noise determination process in step S505. In this example, since the length of the long side 603 of the circumscribed rectangle 602 with respect to the connected pixel block 601 is shorter than the first threshold, the connected pixel block 601 having the circumscribed rectangle 602 is noise. Judged and removed. On the other hand, since the long side 606 of the circumscribed rectangle 605 for the connected pixel block 604 is longer than the first threshold value, the connected pixel block 604 having the circumscribed rectangle 605 is left as a candidate for the boundary region.

  Next, in step S506, the circumscribed rectangle parameter comparison unit 405 compares the circumscribed rectangle with an aspect ratio (long side length / short side length) with a predetermined threshold (second threshold) to determine the first noise. The unit 406 determines from the comparison result whether the circumscribed rectangle is a boundary region candidate. Here, in the noise due to the length of the long side of the circumscribed rectangle in step S505, the length of the long side of the circumscribed rectangle becomes long when the size of the noise is large, such as when there is a large background pattern, and may not be determined as noise. is there. On the other hand, in the noise determination in step S506, it is highly likely that the boundary region of the paper surface is elongated in one direction. The circumscribed rectangle is close to a rectangle, whereas the circumscribed rectangle in the case of noise is close to a square. This property is used.

  Specifically, if the aspect ratio of the circumscribed rectangle (long side length / short side length) exceeds the second threshold, it is determined as a boundary region candidate, and the process proceeds to step S507. Leave connected pixel blocks. On the other hand, if the aspect ratio of the circumscribed rectangle is equal to or smaller than the second threshold, it is determined that there is a possibility of noise, and the process proceeds to the noise determination in step S508.

  FIG. 7 shows a specific example of the noise determination process in step S506. In this example, since the aspect ratio (long side length / short side length) of the circumscribed rectangle 702 with respect to the connected pixel block 701 is larger than the second threshold, it is determined that this is a candidate for the boundary region. Leave connected pixel blocks. On the other hand, the aspect ratio of the circumscribed rectangle 704 with respect to the connected pixel block 703 is equal to or less than the second threshold value, and it is determined that there is a possibility of noise. However, in this determination method, the aspect ratio of the circumscribed rectangle 706 also becomes the second threshold value or less for the connected pixel block 705 in the boundary region extending in the oblique direction. Since there is a high possibility that the connected pixel block 705 extending in the oblique direction is an edge on the paper, it is not yet determined as noise, and the process proceeds to the third noise determination in step S508.

  In step S508, the circumscribed rectangle parameter comparing unit 405 compares the density of the connected pixel blocks in the circumscribed rectangle with a predetermined threshold (third threshold), and the first noise determining unit 406 determines that the circumscribed rectangle is a boundary from the comparison result. It is determined whether or not it is a region candidate. As described above, whether the pixel block extending diagonally is an edge of the paper or noise cannot be determined by the aspect ratio of the circumscribed rectangle, and therefore, determination is performed based on the density of the connected pixel blocks in the circumscribed rectangle.

  This utilizes the following properties: That is, if the circumscribed rectangle is a boundary region corresponding to the edge of the paper surface, there is only a connected pixel block on the diagonal line in the rectangle, so that the pixel density in the rectangle is low. On the other hand, in the case of noise, the pixels are distributed everywhere in the rectangle more widely than at least in the boundary region, so that the pixel density is increased. If the density of the connected pixel block in the circumscribed rectangle is equal to or higher than the third threshold value, it is determined as noise and the process proceeds to step S509, and the connected pixel block is removed. On the other hand, if the density is less than the third threshold, it is determined as a boundary region candidate, and the process proceeds to step S507 to leave this pixel block.

  A specific example of the noise determination process in step S508 is shown in FIG. In this example, since the connected pixel block 701 has already been determined as a boundary region candidate in step S506, it is not subject to processing in step S508. Regarding the connected pixel block 703, since the connected pixel blocks are widely distributed within the circumscribed rectangle 704, the pixel density is high and the pixel block is determined to be noise. On the other hand, the connected pixel block 705 has a low pixel density because the spread of the pixel block within the circumscribed rectangle 706 is only on the diagonal line, and is determined as a boundary region candidate, and this pixel block remains.

  As described above, the edge determined to be noise is removed from the processing target image by the series of processing shown in the flowchart of FIG. 5, and the image is output as an image in which the edge of the paper surface that is a candidate for the boundary region remains. In the case of an image that has undergone this edge detection processing, the boundary region can be correctly detected by the subsequent straight line / line segment output processing.

Line Segment Output Process The line segment output process in step S303 will be described in detail with reference to FIGS.

  FIG. 14 is a diagram illustrating a functional configuration when performing line segment output processing in the image processing apparatus of the present embodiment. Similar to the captured image input unit 401, the captured image input unit 1401 inputs a processing target image. The edge image creation unit 1402 performs the processing from step S502 to step S509. The straight line generation unit 1403 generates a straight line in step S302. The line segment output unit 1404 performs line segment output processing according to the present embodiment. Further, the line segment output unit 1404 can be divided into a plurality of more detailed functional components. Furthermore, the present embodiment is not limited to the above-described configuration. For example, the edge image creation unit 1402 may perform only the edge image creation processing in step S502 as in the edge image creation unit 402 in FIG.

  Next, detailed functional components of the present embodiment will be described. FIG. 8 is a flowchart showing line segment output processing.

  First, in step S801, an edge pixel is selected from the edge pixel group obtained in step S502 based on the straight line group obtained in step S302, and this edge pixel is added to the edge pixel group obtained in step S301. The edge pixel group (edge image) obtained in step S301 is subjected to noise removal in order to generate a straight line with high accuracy. However, in the line segment output process in the present embodiment, line segments are generated based on edge pixels in the vicinity of the straight line, and therefore it is desirable that the edge pixels in the vicinity of the straight line are not removed as noise. Therefore, in step S801, the edge pixel adding unit 1405 adds edge pixels within a predetermined distance (N1) from each straight line to the edge image obtained in step S502 to the edge image obtained in step S301. The obtained edge pixel group is used in the subsequent processing. For example, the number of pixels corresponding to 2 mm is used as the predetermined distance N1.

  An outline of the edge pixel addition processing will be described with reference to FIG. FIG. 13A shows an example of an edge image created by the process of step S502. FIG. 13B shows the edge pixel 1301 left by the process of step S507 and the edge pixel 1302 removed as noise by the process of step S509. An edge image including only the edge pixel 1301 is the edge image obtained in step 301.

  FIG. 13C shows a straight line 1305 generated by the process of step S302. With respect to the straight line 1305, it is estimated that a line segment exists in a portion where an edge pixel exists and a line segment does not exist in a portion where no edge pixel exists. In the vicinity of the straight line 1305 (around the straight line 1305), when the interval between the edge pixels is equal to or greater than a predetermined value n, the straight line 1305 is divided into line segments with the edge pixel as a boundary. When the straight line 1305 is divided, a short line segment 1306 is obtained as shown in FIG. Here, when the edge pixel adding unit 1405 adds the edge pixel 1304 that is within a predetermined distance from the straight line 1305 and removed as noise to the edge image of the edge pixel 1301, and then divides the straight line 1305 into line segments, A correct line segment 1307 is obtained. The process of dividing the straight line into line segments is a part of the line segment output process of the present embodiment performed by the line segment extraction unit 1406 in step S802 described later.

  The line segment output process will be described with reference to FIG. In order to extract a correct line segment, in step S801, a pixel group that is in the vicinity of the straight line generated in step 303 and is removed as noise from the edge image with respect to the edge pixel group obtained in step S302. An added edge pixel group is generated. Next, in step S802, the line segment extraction unit 1406 divides each straight line generated by the straight line generation unit 1403 in step S302 into a plurality of line segments. Specifically, first, an edge pixel is extracted within a predetermined distance (N2) from the straight line from the edge pixel group obtained in step S801. If the distance between the extracted edge pixels is smaller than the predetermined value n, it is determined that they are elements of the same line segment. If the distance between the edge pixels is greater than the predetermined value n, it is determined that they are different line segment elements.

  Here, two edge pixels whose distance is larger than the predetermined value n are defined as an edge pixel pi and an edge pixel pj, respectively. The edge pixel pj and the edge pixel pj are determined to be the end points of different line segments, and a line segment whose end points are coordinates obtained by orthogonally projecting the edge pixels pi and pj into a straight line is generated. The predetermined distance N2 is set to the same value as the predetermined distance N1 determined in step S801, for example. Further, as the predetermined value n, for example, it is necessary to set a value that is equal to or larger than the minimum interval between the paper surfaces in the processing target image, such as a plurality of paper surfaces shown in FIG. .

  Next, in step S803, fitting is performed on edge pixels in the vicinity of the line segment (around the line segment) for each line segment obtained in step S802. The fitting process is a process of generating a line segment for each line segment obtained by the division process and using the edge pixels in the vicinity thereof, and outputting the line segment. Details of the fitting process will be described later with reference to FIGS.

  After performing the processing of step S802 and step S803 for all the straight lines and all the line segments, in step S804, the line segment candidate selecting unit 1412 generates a plurality of line segment candidates from the two line segments to be selected, and the plurality of line segment candidates. One line segment is selected from the line segment candidates. When both the line segment distance and the line segment angle are equal to or less than the threshold value, the line segment candidate selection unit 1412 determines the two line segments as a target of selection processing. This determination is performed for all the line segments obtained by the line segment fitting process in step S803. Here, for example, when it is known that the number of sheets in the image is one, it is necessary to positively select two line segments on the same straight line. In this case, if the threshold of the line segment distance is increased, two line segments arranged in parallel at a distance within the threshold are also subject to selection processing, and an unintended result is obtained. Therefore, as a selection condition, a threshold is given to the line segment connecting the midpoints of both line segments and the angle formed by each of the two line segments, so that only the line segment on the extension line is selected. be able to. A specific example of this selection process will be described later with reference to FIG.

  The line segment selected by the selection process is subjected to the fitting process again in step S805. This is performed for all line segments, and the line segment output process is terminated.

Line Segment Fitting Process Here, the line segment fitting process in steps S803 and S805 will be described in detail with reference to the flowchart of FIG.

  First, in step S901, the reliability calculation unit 1408 uses a calculation formula that determines the reliability of the line segment to be processed (line segment 1) by [line segment length] × [separation degree] / [mean square error]. To calculate. Here, [Line segment length] is the length of the line segment to be processed. [Separation degree] is an index representing how much the luminance of both sides of a line segment is separated. [Mean square error] is an index that represents the degree (density degree) of edge pixels being densely adjacent to a line segment (a predetermined range around the line segment). Here, [Separation degree] is calculated by the following equation (1).

Here, N ₁ is the number of pixels in region 1, N ₂ is the number of pixels in region 2, m ₁ is the average luminance value of region 1, m ₂ is the average luminance value of region 2, and m is the average luminance value of region 1 to region 2. s _i is the luminance value of the pixel i In the expression (1), the region 1 is one region having a width w having a line segment as one side, and the region 2 has a width w that does not overlap the region 1 having the line segment as a side. The other area. The reliability can also be calculated when the input image data is multivalued data (for example, RGB ternary data). For example, in a multi-value space (RGB space), the value obtained by projecting all pixel values onto a straight line passing through the average value of region 1 and the average value of region 2 is replaced with a luminance value, and the degree of separation is calculated using Equation 1. Then, the reliability may be calculated.

  [Mean square error] is a value obtained by further taking the square root of the average obtained by squaring the distance between edge segments within the distance D from the line segment. In the present embodiment, w and D are set to the same value as the predetermined distance N1 used in the edge pixel group extraction in step S801.

  That is, the reliability as a line segment indicating the boundary of the region is calculated based on the relationship between the distribution of edge pixels near the line segment and the image feature amount of the two regions having the line segment as a boundary. In addition, the reliability may be calculated based only on the distribution of edge pixels in the vicinity of the line segment, or only on the relationship between the image feature amounts of the two regions having the line segment as a boundary. The distribution of edge pixels in the vicinity of a line segment can be expressed by using, for example, a mean square error that is a degree of density of edge pixels in a predetermined range around the line segment. In addition, the relationship between the image feature amounts of two regions having a line segment as a boundary can be expressed by using, for example, a degree of separation that is an index of how much the luminance on both sides of the line segment is separated.

  FIG. 10 shows a conceptual diagram of the degree of separation. In FIG. 6A, the luminance histograms of the regions 1002a and 1002b having the line segment 1001 as a boundary are 1003a and 1003b. Similarly, histograms obtained from the respective regions in FIG. 6B are shown in 1004a and 1004b. In this example, the degree of separation is higher in FIG. 10 (a) than in FIG. 10 (b).

  The reliability of a line segment is an index that increases when the line segment is long, the luminance changes greatly on both sides with the line segment as a boundary, and the edge pixels are close to the line segment. The reliability is not limited to the equation (1), and a modification thereof or another equation may be used. For example, [line segment length] can be replaced with [number of edge pixels within distance D]. [Separation degree] is replaced with [maximum luminance difference (or average luminance difference) between region 1 and region 2]. [Mean square error] can be replaced with [Total sum of distances between edge pixels and line segments]. In addition, [mean square error] is replaced with [dispersion in luminance gradient direction of edge pixel group within distance D]. Further, it is possible to modify the line segment reliability by multiplying [the sum of luminance gradient intensities of edge pixel groups within the distance D] as a coefficient, and the present invention is not limited to the above-described modification method.

  The reliability calculation unit 1408 can be further divided into a line segment length calculation unit, a separation degree calculation unit, and a mean square error calculation unit. Each of the line segment length, the degree of separation, and the mean square error may be weighted. For example, regarding the two line segments to be compared, when the difference between the line segment length and the mean square error is small, the degree of separation calculated by the degree of separation calculation unit may be used as the reliability.

  Returning to FIG. 9, in step S <b> 902, the edge pixel acquisition unit 1409 acquires the edge pixel group E <b> 1 within a predetermined distance (D) from the line segment 1. Next, in step S903, the line segment acquisition unit 1410 newly obtains a line segment 2 from the edge pixel group E1. First, a straight line is generated by applying processing such as principal component analysis to the edge pixel group E1. Principal component analysis is a method for obtaining a direction in which the variance of a point group is maximized. Next, a line segment whose end point is an intersection of a straight line generated from the edge pixel group E1 and a circumscribed rectangle including the edge pixel group E1 is acquired as a line segment 2.

  Next, in step S904, the reliability of line 2 is obtained in the same manner as the reliability of line 1 obtained in step S901. In step S905, the line segment selection unit 1411 compares the reliability of the line segment 1 with the reliability of the line segment 2, and selects the larger one as the line segment A. Here, when the line segment 1 is selected as the line segment A, the process proceeds to step S906, and the predetermined distance D is updated to a value equal to or less than D. For example, D = D / 2. When the line segment 2 is selected as the line segment A, the predetermined distance D is not updated to a small value, and the process directly proceeds to step S907.

  In step S907, an edge pixel group E2 having a predetermined distance D or less from the line segment A is acquired. When the line segment A is the line segment 1, the edge pixel group E2 is a point group that is completely included in the edge pixel group E1. On the other hand, when the line segment A is the line segment 2, the edge pixel group E2 becomes a point group overlapping with the edge pixel group E1. If the edge pixel group E2 obtained in step S907 is equal to the edge pixel group E1, it is considered that the process has converged, and the line segment fitting process is terminated. On the other hand, if the edge pixel group E2 and the edge pixel group E1 are different, the process proceeds to step S908, the edge pixel group E2 is set as a new edge pixel group E1, the line segment A is set as a new line segment 1, and the process returns to step S903. continue processing.

  The line segment fitting process shown in the flowchart of FIG. 9 is merely an example, and the line segment fitting process is not limited to this example. For example, in step S902 and step S907, by acquiring only edge pixels having a luminance gradient in a direction similar to the angle of line segment 1, the accuracy of step S903 can be improved.

  Further, the line segment A selected in step S905 may be output as it is as a result of the fitting process, or the calculation may be forcibly terminated at a predetermined number of loops, and the line segment A at that time may be output. Also, a plurality of values Di of the predetermined distance D are prepared in advance, and the edge pixel group Ei and the line segment i are obtained for each predetermined distance Di at the time of Step S902 and Step S903, and the line segment i having the maximum reliability is selected. May be. Alternatively, a method other than principal component analysis may be used in step S903. For example, two points in contact with the circumscribed rectangle or two points having the maximum distance may be set as the end points. Further, a known method such as the least square method, the Hough transform, or RANSAC may be used. Further, all of these may be used simultaneously to obtain a plurality of line segments, and either one may be selected, or all line segments whose reliability is equal to or greater than a threshold may be output.

  FIG. 11 shows a specific example of the line segment fitting process. In FIG. 11A, from the edges 1101a to 1101c obtained from images obtained by imaging three sheets of paper and the pixels 1102 that are linear noise but could not be removed, a straight line 1103 is generated by the straight line generation processing in step S302. Is obtained. Here, the edge 1101c is one edge corresponding to the upper side of the paper surface C, but the two edges 1101a and 1101b correspond to the upper side of the paper surface A and the paper surface B, respectively, due to the influence of ambient light or the like. Is the edge.

  When the straight line 1103 is divided based on the edge pixels in the vicinity of the straight line 1103 (a predetermined range around the line segment) by the process of step S802, line segments 1104a to 1104c shown in FIG. 11B are obtained. Since the straight line 1103 is obtained from a plurality of paper edge pixels and edge pixels corresponding to noise, the line segments 1104a to 1104c are simply the result of dividing the straight line 1103, both of which are shifted in position and inclination from the upper side of each paper surface. is there. Line segment fitting processing is performed on these line segments 1104a to 1104c.

  FIG. 11C shows the result of applying the processing of step S902 and step S903 to each of the line segments 1104a to 1104c (line segment 1). A line segment 1105b and a line segment 1105c suitable for the upper side of the paper surface B and the paper surface C were obtained as the line segment 2 from the edge 1101b and the edge 1101c, respectively. On the other hand, the line segment 1105a is a line segment (line segment 2) that is displaced from the upper side of the paper surface A due to the influence of the linear noise pixel 1102. Here, the processing of step S905 is performed, and the reliability of each of the line segment 1104a and the line segment 1105a, the line segment 1104b and the line segment 1105b, and the line segment 1104c and the line segment 1105c is calculated and compared.

  As a result, the line segment 1104a, the line segment 1105b, and the line segment 1105c are each selected as a line segment with high reliability. Since the line segment 1105b and the line segment 1105c indicate the boundaries of the paper area, the process converges and the line segment fitting process ends. On the other hand, since the line segment 1104a selected as the line segment 2 is a line segment obtained by dividing the straight line 1103, the process proceeds to step S906, the predetermined distance D is updated to a smaller value, and the edge pixel group is obtained again.

  As shown in FIG. 11D, the predetermined range before the update around the line segment 1104a is a range within the predetermined distance D from the line segment 1104a with the line segment 1104a as the center line. FIG. 11E is a diagram showing the updated predetermined range around the line segment 1104a. A predetermined range 1106 corresponding to the predetermined distance D and a smaller predetermined range 1107 corresponding to the updated predetermined distance D / 2 are shown. By setting the predetermined distance to D / 2, the linear noise pixels 1102 are removed, and an edge pixel group including only the edge 1101a is obtained. This edge pixel group is acquired as the edge pixel group E2 in step S907, and the process proceeds to step S908 and step S903 to perform processing such as principal component analysis. Then, as shown in FIG. 11F, two segmented edges 1101a are connected to obtain a line segment 1108a indicating the boundary of the correct paper surface area.

  In FIG. 11 (e), as a method of reducing a predetermined range around the line segment 1104a (a predetermined range in the normal direction of the line segment), a predetermined distance D from the line segment 1104a on both sides of the line segment 1104a as a boundary. An example in which is changed to a predetermined distance D / 2 is shown. However, with this method, if the linear noise pixel 1102 cannot be removed, the line segment 1108a may not be obtained correctly, and a line segment with low reliability may be obtained.

  Therefore, when the reliability of the line segment obtained by the above-described method is lower than the line segment 1104a, the position of the predetermined range whose size is reduced is moved in the normal direction of the line segment 1104a, and is in the moved predetermined range. The edge pixel group is acquired again, and a line segment is generated. By comparing the reliability of the generated line segment with the line segment 1104a, linear noise is removed. Note that the movement within a predetermined range with a reduced size is performed a plurality of times, and the process is repeated until a line segment with higher reliability than the line segment 1104a is found.

  It should be noted that the line segment fitting process of the present embodiment can also be applied when there is only one sheet. For example, as shown in FIGS. 11 (g) to 11 (i) and FIGS. 11 (d) to 11 (f), the above-described method can be similarly applied even when there is no paper surface B and paper surface C but only paper surface A.

Selection of Line Segment Candidate Line segment candidate selection processing in step S804 will be described with reference to FIG.

  FIG. 12A shows two line segments 1201 and 1202, which are two line segments before selection processing, end points 1201a and 1201b of the line segment 1201, and end points 1202a and 1202b of the line segment 1202. .

  In the line segment candidate selection process, as shown in FIGS. 12B to 12E, first, line segment candidates 1203 to 1206 having the end point 1201a or 1202a and the end point 1201b or 1202b as end points are obtained. Here, the line segment candidate 1203 and the line segment candidate 1204 are the same as the original line segment 1202 and the line segment 1201. A line segment candidate 1205 and a line segment candidate 1206 are newly generated line segments. Next, the reliability similar to step S901 is calculated | required about these candidate line segments, and it outputs as a result of having selected the line segment candidate with the highest reliability.

  When the line segment output here is the line segment candidate 1205 or the line segment candidate 1206, since these are newly generated, the line segment fitting is performed again in step S <b> 805. Further, the line segment candidates are not limited to the line segments shown in FIGS. 12B to 12E. For example, the midpoints of the end points 1201a and 1202a and the midpoints of the end points 1201b and 1202b are newly determined, and the end points are defined as the end points. Line segment candidates may be obtained.

  A plurality of line segments may be output from the boundary (same side) of the same paper region due to slight distortion of the paper surface or shadow of the paper surface caused by ambient light. On the other hand, by performing line segment candidate selection processing, a new line segment is selected from two adjacent line segments, and a line segment indicating the boundary of the correct paper area can be output. .

Quadrangle Generation Process In the rectangle generation process in step S304, a set of rectangles is generated from the line segment group obtained in step S303.

  First, the angles of all line segments are obtained, line segments whose angles are 0 degree or more and less than 45 degrees are classified into horizontal line segments (hereinafter referred to as horizontal line segments), and line segments that are 45 degrees or more and less than 90 degrees are vertically aligned. To the line segment (hereinafter referred to as the vertical line segment). Next, an arbitrary line segment is selected from the horizontal line segment group as a first horizontal line segment, and a second horizontal line segment that is separated by a certain distance or more is selected. Subsequently, a vertical line segment that is substantially orthogonal to the first horizontal line segment (for example, 90 degrees ± 3 degrees) is selected from the vertical line segment group as the first vertical line segment, and the second vertical line that is more than a certain distance away from the vertical line segment is selected. Select a line segment. Finally, a quadrangle whose vertexes are four intersections obtained by extending the first and second horizontal line segments and the first and second vertical line segments is output, and this is searched for a combination of all line segments. This creates a rectangle.

  In the present embodiment, a quadrangle is described as a detection target, but the present invention is not limited to this. For example, when an N-gon (N is an odd number) is an object to be detected, the line segment is classified into N at 90 / N degree increments, and an arbitrary line segment is acquired from each classified group and combined to form the N-gon. It is possible to generate. In the case of M square (M is an even number), it is possible to generate an M square by classifying the line segment into 90 / M degree increments, and obtaining and combining two arbitrary line segments from each classified group. It is.

Paper Area Specifying Process Hereinafter, the paper area specifying process in step S305 will be described in detail with reference to FIGS. Here, for explanation, a quadrilateral paper area candidate is described as an example, but the shape of the area candidate is not limited to a quadrilateral, and may be a polygon composed of corners and sides. Further, the region candidate is not limited to the paper surface, but may be a sheet shape made of a material such as plastic.

  FIG. 16 is a diagram illustrating a functional configuration when performing the paper surface area specifying process in the image processing apparatus of the present embodiment. Similar to the captured image input unit 401, the captured image input unit 1601 inputs a processing target image. The paper area candidate input unit 1602 is a functional block for inputting paper area candidates. The paper area candidate input unit 1602 of this embodiment includes an edge image creation unit 1603, a straight line generation unit 1604, a line segment output unit 1605, and a quadrangle generation unit 1606.

  The edge image creation unit 1603 performs the processing from step S502 to step S509 in the same manner as the edge image creation unit 1402. The straight line generation unit 1604 generates the straight line in step S302 as with the straight line generation unit 1403. Similar to the line segment output unit 1404, the line segment output unit 1605 performs line segment output processing. The quadrangle generation unit 1606 performs the process of step S304 using the output line segment, and generates a quadrilateral that is a paper area candidate for the paper area.

  The information output unit 1607 outputs information regarding the paper area candidate generated by the quadrangle generation unit 1606. An area specifying unit 1608 performs a paper area specifying process.

  Next, the flow of the paper area specifying process will be described with reference to FIG. First, in step S1501, the processing from step S1502 to step S1507 is repeated with the paper region candidates O1 to On generated as squares in step S304 sequentially as the paper region candidate O to be processed. Since the processing for each paper area candidate O is independent, the finally obtained result does not vary depending on the processing order. Therefore, the processing of each paper area candidate O may be performed in parallel.

  In step S1502, the vicinity of the side of the paper region candidate O is divided into m pieces to create divided regions B1, B2,. Here, the division of the region near the side of the paper region candidate O will be described with reference to FIG.

  FIG. 17 is a diagram for explaining the division of the region near the side of the paper region candidate O generated by the quadrangle generation unit 1606. As shown in FIG. 17A, the paper surface area candidate is a quadrangle composed of four line segments 1701, 1702, 1703, and 1704.

  FIG. 17B shows an example of dividing points obtained by dividing the four line segments 1701, 1702, 1703, and 1704 shown in FIG. In this embodiment, the dividing points 1705 to 1716 are set so that the line segments 1701 and 1703 corresponding to the long side of the quadrangle 1601 are equally divided into five and the line segments 1702 and 1704 corresponding to the short side are equally divided into three equal parts. Further, an example in which a total of 16 division points are created by combining the division points 1717, 1718, 1719, and 1720 corresponding to both ends of each side, that is, the corners of the quadrangle 1601, is shown. Note that the dividing points shown in FIG. 17 are merely examples, and each side may be divided into different numbers, or may be divided into the same number on all sides. In addition to the method of dividing the sides equally, the dividing points may be determined at regular intervals from the ends of the sides.

  Then, as shown in FIG. 17C, square divided areas 1721 to 1736 centering on each dividing point are created. Each divided area has its center on a quadrilateral side that is a paper area candidate, and the inside and outside of the quadrangular side are configured by dividing the inside and the outside by the quadrangular side. This divided area is merely an example. For example, the shape of the divided area may be a rectangle instead of a square, or may be a circle or other shapes. Further, the size of each divided region may not be constant, and may be not more than a predetermined size. Further, a predetermined size having the highest evaluation value may be obtained in accordance with an evaluation value of a divided region described later, and the size of each divided region may be changed to the predetermined size or less.

  In step S1503, an evaluation value for obtaining the certainty (accuracy) of the paper area candidate is calculated for the divided area created in step S1502. The evaluation value calculation method for each divided area will be described later with reference to FIG.

  In step S1504, a probability (accuracy) score of the entire paper area candidate is calculated based on the evaluation value for each divided area calculated in step S1503, and the process proceeds to step S1505. The score of the paper region candidate is obtained by calculating the average of the evaluation values of the divided regions located on the sides for each of the four sides and adding the average of the evaluation values of the divided regions located on the corners.

  In the example of the divided area shown in FIG. 17, the divided areas 1721 to 1732 are located on the sides, and the divided areas 1733 to 1736 are located on the corners. The average evaluation value for each side means that the average of the divided areas 1721 to 1724, the average of the divided areas 1725 and 1726, the average of the divided areas 1727 to 1730, and the average of the divided areas 1731 and 1732 It is.

  If the paper area candidate is a correct paper area, the degree of separation shows a high value due to the color difference between the inside of the paper area candidate, that is, the paper surface and the outside thereof. The method of adding the average values of the corners and sides described in step S1504 is an example for describing the present embodiment, and the higher the degree of separation of the divided regions, the higher the accuracy of the paper region candidate. Since it is essential, the specific method for calculating the score may be changed. As an example of a change in the method for calculating the score, a calculation result in the middle can be weighted and calculated. For example, when the score is calculated in step S1504, the degree of separation of the divided areas at the corners may be weighted, or the average value may be weighted. In addition, a weight may be given to the degree of separation of each divided region in the side, or an average value thereof may be weighted. As a weighting method, a predetermined coefficient can be multiplied by the degree of separation or the average value of the degree of separation.

  Note that the degree of separation according to the present embodiment can be calculated by Expression (1), but is not limited to Expression (1). The degree of separation of the divided regions 1721 to 1732 is output by the information output unit 1607 as information indicating the relationship between the image feature amounts on both sides with the line segment corresponding to the side as a boundary. Further, the degree of separation of the divided regions 1733 to 1736 is output by the information output unit 1607 as information indicating the relationship between the inner and outer image feature amounts of the region candidates.

  In step S1505, it is determined whether the score of the target paper area candidate O is equal to or greater than the threshold value. If the score is equal to or greater than the threshold value, the process proceeds to step S1506. On the other hand, if it is less than the threshold value, the process advances to step S1507, and the target paper area candidate O is not a paper area, is deleted from the paper area candidate group, and the process advances to step S158. In the present embodiment, this threshold value is 1.80, but this threshold value is only an example, and may be another fixed value or a value that can be adjusted according to the type of the actual processing target.

  In steps S1505 to S1507, it is determined whether or not it is a paper surface using a threshold value. If it is clear that there is only one paper region in the input image, a rectangular region candidate having the highest score is obtained. You may leave only.

  If the processing for all the paper area candidates is completed in step S1508, the paper area specifying process shown in FIG. 15 is ended. If there are unprocessed paper area candidates remaining, the process returns to step S1501, The processing of the paper area candidate is started.

Division Area Evaluation Value Calculation Processing Hereinafter, the division region evaluation value calculation processing in step S1503 will be described with reference to the flowchart of FIG.

  First, in step S1801, the divided areas B1 to Bm created in step S1502 are sequentially set as the divided areas B to be processed, and the processes in steps S1802 to S1803 are repeated. Since the processing for each divided region B is independent, the finally obtained result does not vary depending on the processing order. Therefore, the processing of each divided region B may be performed in parallel.

  In step S1802, the degree of separation between pixel values (for example, luminance values) in the inner area and the outer area of the divided area B is calculated. The above formula (1) may be used to calculate the degree of separation. Here, the degree of separation is an index indicating how much pixel values (for example, luminance values) on both sides of a line segment with the line segment as a boundary are separated. If the paper area candidate O is a correct paper area, the degree of separation is calculated to be high based on the difference in pixel values (eg, the difference in luminance values) between the paper area and the back area (eg, the original and the original table). In step S1803, the degree of separation obtained in step S1802 is set as the evaluation value of the divided region B, and the process proceeds to step S1804.

  Conventionally, the degree of separation is obtained for the entire region in the vicinity of the side of the paper region candidate O. Generally, in a wide region, the distribution of the luminance histogram becomes wide due to the influence of ambient light or the like. Therefore, as the degree of separation between the two areas in the wide area, the difference between the pixel values (luminance values) of the two areas is small, so that the histograms of the two areas overlap and a value with a small degree of separation is calculated. . Therefore, in this embodiment, by dividing the wide area into small areas and calculating the degree of separation so that the correct paper area candidate can be specified even when the difference between the pixel value (luminance value) between the background and the paper is small. The correct paper area candidate can obtain a higher degree of separation.

  Further, the size of the divided area may be changed in order to obtain a higher degree of separation. For example, in a portion where the margin in the paper is narrow, extra pixels such as characters may be included depending on the size of the divided area, and the separation degree may take a small value. In that case, it is only necessary to change the size of the divided area and adopt the higher degree of separation as the evaluation value.

  In step S1804, if all the divided areas have been processed, the evaluation value calculation process shown in FIG. 18 is ended. If there is an unprocessed divided area, the process returns to step S1801, and the next divided area is determined. Start processing the region.

Evaluation Value Calculation Example FIG. 19 shows an example of evaluation values calculated in step S1503 in FIG. 15 for the paper area candidate shown in FIG. The result of calculating the degree of separation of the divided regions 1721 to 1732 in steps S1802 and S1803 is described in the evaluation value column of Table 1901. Moreover, the result of calculating the average of the evaluation values of each side is the result of calculating the average of the evaluation values of the corners 1733 to 1736 in the column of the average evaluation value of each side as the average of each side. It is listed in the value column.

  Based on the average evaluation value shown in FIG. 19, the score of the accuracy of this paper area candidate is calculated to be 1.82 in step S1504. This score is output from the information output unit 1607. In step S1505, it is determined that the score is equal to or greater than the threshold value, and the process advances to step S1506. In step S1506, based on the score determination result, the area specifying unit 1608 specifies the paper area candidate shown in FIG. 17 as the paper area or the attention area.

  In the following, an example of evaluation value calculation for the same input image will be shown by taking a paper area candidate different from the paper area candidate shown in FIG. 17 as an example. The paper surface area candidate shown in FIG. 20A is composed of the upper end 2001 of the black belt portion on the paper surface, the line segment 2002 on the three sides of the paper surface, the line segment 2003 and the line segment 2004, and is selected as the paper surface. It is not preferable. FIG. 20B shows an example in which the four line segments 2001 to 2004 shown in FIG. 20A are divided in step S1502 of FIG. In the same figure, it was divided into divided areas 2004-2013 on the side and divided areas 2014-2017 on the corner.

  FIG. 20C is an enlarged view of the outer region of the corner divided region 2014. The outer area of the divided area 2014 is composed of a background 2018 and a paper surface 2019. When the luminance histogram for the outer region is obtained, the width of two peaks generated by the pixel group constituting the background 2018 and the paper surface 2019 is widened, so that the variance value of the outer region is increased and the outer region and the inner region are separated. Indicates a low value. That is, the evaluation value of the corner portion divided area 2014 becomes small. The same applies to the divided areas 2015.

  FIG. 21 shows an example of evaluation values calculated in step 1502 and step S1503 in FIG. 15 for the divided areas of the paper area candidate in FIG. In Table 2101, the evaluation values of the divided areas 2014 and 2015 are low. Based on the processing result shown in FIG. 21, the score of the accuracy of this paper area candidate is calculated to be 1.72 in step S1504. This value is determined not to satisfy the threshold value in step S1505, and in step S1507, the paper area candidate is deleted as an area that is not correct.

  A specific example of the paper area candidate specifying process will be described with reference to FIGS. 22A and 22B. When a gradation occurs in the entire captured image due to the influence of ambient light as shown in FIG. 22 (a), the method for obtaining the degree of separation between the foreground area and the entire background area can be performed both inside and outside the paper area candidate. The dispersion value becomes a high value due to ambient light, and the separation degree between the inside and the outside is low. In the configuration and method of the present embodiment, dividing each side into small areas shows a high value of the internal / external separation degree of the paper area candidates in each small area.

  FIG. 22B shows a case where a paper area where a colored (black or other color) band area 2201 is present is photographed. Even in this case, since there is a color difference between the inside and outside of each divided area in the colored band area 2201 portion of the correct paper area candidate, the inside / outside separation degree of each divided area shows a high value. Therefore, a high score is calculated for the region candidate.

  In addition, as shown in FIG. 22B, a paper area including a colored (black or other color) block 2210 may be photographed. Even in the paper area candidate generated from the block 2210 in the captured image, the degree of separation between the inner side and the outer side shows a high value. In this case, the areas of the respective paper area candidates are further calculated and compared, and the paper area candidate having a relatively small area is deleted as a non-correct area. As a method for comparing the calculated areas of the respective paper area candidates, for example, an area ratio with respect to the largest paper area candidate can be obtained, and a paper area candidate whose area ratio is equal to or smaller than a threshold value can be determined as an object to be deleted.

  According to the present embodiment, in each divided region obtained by dividing the peripheral region of each line segment constituting the paper region candidate, based on the degree of separation of the histograms of the pixel values (for example, luminance values) outside and inside the paper region candidate. Is calculated as a score. Therefore, there is an effect that the correct paper area candidate can be specified as the paper area or the attention area without selecting an incorrect paper area candidate that is not an actual paper area.

<Second Embodiment>
In the first embodiment, the paper region candidate score is calculated and the paper region candidate is selected by determining whether or not the threshold value is equal to or greater than the threshold value. However, in the present embodiment, the relative scores of the plurality of paper region candidate candidates are selected. A paper area candidate is selected from the value. For example, if it is known in advance that the input image includes only N paper areas, the top N paper area candidates in the scores of all paper area candidates are identified as the paper area or the attention area without using a threshold value. To do. Other processes in the present embodiment are the same as those in the first embodiment, and thus the description thereof is omitted.

  In addition, since the two sheets of paper overlap each other, if the user does not want to select the hidden paper, only the one with the higher score may be selected for the two paper region candidates whose areas overlap.

  FIG. 23 shows an example in which two sheet areas overlap. The paper area candidate 2301 is partially hidden by the paper area candidate 2302. A divided area 2303 is an example of a divided area set by the processing described in the first embodiment with respect to the paper area candidate 2301. In this divided area 2303, since both the inner area and the outer area of the paper area candidate 2301 are the paper face in the paper area candidate 2202, the separation degree is a low value. Also, the entire paper area candidate 2301 is also hidden in the paper area 2302 in both the internal area and the external area, so that the pixel values are not uniform and the variance value is high, and the separation degree of the entire paper area candidate is low. Become. As a result, the score of the paper area candidate 2301 is lower than that of the paper area candidate 2302. In this case, the area specifying unit 1608 specifies only the paper area candidate 2302 having the highest score as the paper area or the attention area.

  When the number of paper areas or attention areas to be specified by the user is known in advance, according to the present embodiment, a predetermined number of paper candidate areas are specified based on the score ranking of the plurality of paper candidate areas. Further, it is not possible to specify a paper area candidate that is partially hidden by overlapping as shown in FIG.

<Third Embodiment>
In the first embodiment, a probability (accuracy) score as a paper area is calculated based on the inner and outer separation degrees of the paper area candidate areas of each divided area. In the present embodiment, when the pixel values of the background area (such as the document table) of the imaged paper are uniform, the variance value is used as a score index. Other processes in the present embodiment are the same as those in the first embodiment, and thus the description thereof is omitted.

  FIG. 24 is a flowchart showing the process of calculating the evaluation value of the divided area in step S1503 in this embodiment.

  In step S2401, the divided areas B1 to Bm are set as the divided areas B to be processed, the process proceeds to S2402, and the processes up to S2406 are repeated.

  Since the subsequent processing is determined by the position of the divided area B, in step S2402, it is first determined whether the divided area B is located at a corner. If it is determined that the divided area B is located on the side, the processes in steps S2403 and S2404 are performed. If it is determined that the divided area B is located in the corner, the processes in steps S2405 and S2406 are performed.

  Since the process of step S2403 and step S2404 is the same as the process of step S1802 and step S1803, description is abbreviate | omitted here. In step S2405, the information output unit 1607 calculates and outputs the variance value σ of the outer area of the paper candidate area in the divided area at the corner using the following equation (2).

Here, N is the number of pixels, m is the average luminance value of the region, s _i is the luminance value of the pixel i, and the luminance value is used to calculate the variance value σ in Equation (2), but is not limited to the luminance value. When the input image is multi-valued (for example, RGB values), the principal component analysis is performed on all the pixel values in the multi-valued space (RGB space), and all the pixel values are calculated with respect to the main axis obtained from the first principal component. The variance value σ can also be calculated by applying the value obtained by projection to the equation (2) instead of the luminance value.

  In step S2406, the variance value of the outer area obtained in step S2405 is set as the evaluation value of the divided area B at the corner, and the process proceeds to step S2407.

  In step S2407, if the process is completed for all the divided areas B, the process ends. If an unprocessed area remains, the process returns to step S2401.

  In the present embodiment, the information output unit 1607 calculates and outputs a score using the following equation (3) in the score calculation process in step S1504.

here,
η _m is the average of the inside / outside separation degree of each side of the divided region obtained in S1503 σ _m is the average of the external variance of each corner of the divided region obtained in S1503 Equation (3) is an example of the score calculation process of this embodiment It is essential that the accuracy of the paper region candidate increases as the degree of separation of the divided regions on each side increases, and the accuracy of the paper region candidate increases as the variance value outside the corner decreases. ) May be given a different weight. Moreover, you may obtain | require a score by other methods, such as calculating a score only by the degree of separation or a dispersion value.

  In step S1505, the score output from the information output unit 1607 is determined. On the basis of the determination result, in step S1506, the area specifying unit 1608 uses the paper area candidate 2501 shown in FIG. 25 as the paper area or attention area. Identify.

  FIG. 25A shows an example of an input captured image and a paper area candidate obtained from the captured image. A paper area candidate 2501 is a paper area candidate with rounded corners. In the quadrangle generation processing in step S304, since the line segments that are candidates for the four sides are expanded to generate a quadrangle, such a quadrangle is formed as a paper surface candidate even if the corner has a round paper surface.

  The divided area 2502 is a divided area located at the corner of the paper area candidate 2601 generated in step S1502.

  FIG. 25B is an enlarged view of the divided area 2502. In the divided area 2502, there are an inner area 2503 and an outer area 2504 of paper area candidates. In the case of a paper surface with rounded corners in this way, there may be a case where only a part of the actual paper surface enters the inner region 2503 of the corner divided region, or there is no case. In that case, since there is no difference in the pixel dispersion values in the inner and outer regions, a low separation value is obtained. With the method described in the first embodiment, the score of the paper region candidate is low even in the correct paper region. Become a value. In the method using the variance value, if the background pixel value is uniform, the variance value is low, and the score of the paper area candidate can be calculated high.

  As described above, according to the present embodiment, by using the fact that the pixel values of the background region are uniform and using the variance value of the outer region of the corner of the paper region candidate, the paper surface with rounded corners is used. In addition, it is possible to accurately detect the paper region.

<Other embodiments>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101 Image input unit 102 CPU
103 RAM
104 Storage unit

Claims (17)

An input means for inputting a region composed of a plurality of line segments as a region candidate;
First information indicating the relationship between image feature amounts on both sides of the line segment as a boundary in a plurality of regions having a predetermined size or less in each line segment constituting the region candidate, and a predetermined size at each corner of the region candidate Output means for outputting the second information indicating the relationship between the image feature amount inside and outside the area candidate in the area below
Using the first information and the second information corresponding to each of the plurality of regions, specifying means for specifying the region candidate as a region of interest;
An image processing apparatus comprising:   2. The image according to claim 1, wherein the first information is a first degree of separation of pixel values on both sides with the line segment as a boundary in each of the plurality of regions in each line segment. Processing equipment. The image processing apparatus according to claim 2 , wherein the specifying unit specifies the region candidate as a region of interest based on an average value of the first degree of separation. The image processing apparatus according to claim 1 , wherein the second information is a second degree of separation of pixel values inside and outside the area candidate in the area at each corner. The image processing apparatus according to claim 4 , wherein the specifying unit specifies the region candidate as a region of interest based on the first information and the average value of the second degree of separation. The first information is a first degree of separation of pixel values on both sides with the line segment as a boundary in each of the plurality of regions in the line segments, and the second information is the first information at the corners. A second degree of separation of pixel values inside and outside the region candidate in a region,
The specifying unit, based on the average value of the second separation and the average value of the first degree of separation, the image processing apparatus according to claim 1, wherein the identifying the area candidate region of interest . The specifying unit specifies the region candidate as a region of interest based on the average value of the first degree of separation weighted by the average value of the second degree of separation and the average value of the second degree of separation. The image processing apparatus according to claim 6 . The specifying unit specifies the region candidate as a region of interest based on the average value of the first degree of separation and the average value of the second degree of separation weighted by the average value of the first degree of separation. The image processing apparatus according to claim 6 . The specifying means, any one of the claims 6 to 8, wherein the identifying the region candidate said first separation and evaluation values obtained from the second separation is equal to or higher than the threshold as a region of interest The image processing apparatus according to item. When there are a plurality of region candidates input by the input unit, the specifying unit sets a predetermined number of the region candidates as a region of interest based on the evaluation values obtained from the first degree of separation and the second degree of separation. the image processing apparatus according to any one of claims 6 to 8, wherein the identifying. An input means for inputting a region composed of a plurality of line segments as a region candidate;
First information indicating the relationship between image feature amounts on both sides of the line segment as a boundary in a plurality of regions having a predetermined size or less in each line segment constituting the region candidate, and a predetermined size at each corner of the region candidate and output means for outputting and a third information indicating the image feature amount of the outside of the region candidate in the following areas are,
Using the first information and the third information corresponding to each of the plurality of regions, the specifying means for specifying the region candidate as a region of interest ;
An image processing apparatus comprising: The image processing apparatus according to claim 11 , wherein the third information is a variance value of pixel values outside the area candidate in the area at each corner. The input means includes
Creating means for creating an edge image based on edge pixels extracted from a captured image obtained by imaging a document;
Straight line generating means for generating a straight line from the edge pixels;
Line segment output means for outputting a line segment indicating a boundary of the document based on the straight line generated from the straight line generation means;
Generating means for generating the region candidates from the line segments generated by the line segment output means;
The image processing apparatus according to any one of claims 1 to 1 2, characterized in that it further comprises a. An input step of inputting a region composed of a plurality of line segments as a region candidate;
First information indicating the relationship between image feature amounts on both sides of the line segment as a boundary in a plurality of regions having a predetermined size or less in each line segment constituting the region candidate, and a predetermined size at each corner of the region candidate An output step for outputting second information indicating the relationship between the inner and outer image feature amounts of the region candidates in the region below
Using the first information and the second information corresponding to each of the plurality of regions, a specifying step of specifying the region candidate as a region of interest;
An image processing method comprising: An input step of inputting a region composed of a plurality of line segments as a region candidate;
First information indicating the relationship between image feature amounts on both sides of the line segment as a boundary in a plurality of regions having a predetermined size or less in each line segment constituting the region candidate, and a predetermined size at each corner of the region candidate An output step of outputting second information indicating a relationship between the inner and outer image feature amounts of the region candidates in the region below
Using the first information and the second information corresponding to each of the plurality of regions, a specifying step of specifying the region candidate as a region of interest;
A program that causes a computer to execute. An input step of inputting a region composed of a plurality of line segments as a region candidate;
First information indicating the relationship between image feature amounts on both sides of the line segment as a boundary in a plurality of regions having a predetermined size or less in each line segment constituting the region candidate, and a predetermined size at each corner of the region candidate An output step of outputting third information indicating an image feature amount outside the area candidate in the area below
Using the first information and the third information corresponding to each of the plurality of regions, a specifying step of specifying the region candidate as a region of interest;
An image processing method comprising: An input step of inputting a region composed of a plurality of line segments as a region candidate;
First information indicating the relationship between image feature amounts on both sides of the line segment as a boundary in a plurality of regions having a predetermined size or less in each line segment constituting the region candidate, and a predetermined size at each corner of the region candidate An output step of outputting third information indicating an image feature amount outside the region candidate in the region below
Using the first information and the third information corresponding to each of the plurality of regions, a specifying step of specifying the region candidate as a region of interest;
A program that causes a computer to execute.

Images