JP2004310726A - Image inspection method, image inspection apparatus, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enhance the inspection accuracy even when an original protrudes from an imaging range in an image inspection apparatus which inspects goodness/badness of a pick-up image by comparing the pick-up image with a reference image. <P>SOLUTION: An edge valid part linear expression calculation part 126 performs edge valid part linear expression calculation processing about each of vertical and horizontal sides. An original vertex setting processing part 132 specifies vertex coordinates of the original in the pick-up image by referring to the calculated linear expression. A protruding location judgment processing part 134 judges whether or not the vertex coordinates protrude from the imaging range. A boundary part specification processing part 136 specifies a boundary part with the imaging range about an original vertex whose vertex coordinates are judged to protrude from the imaging range. A non-inspection region setting part 138 demarcates a non-inspection region based on coordinate information indicating the boundary part and coordinate information of the protruding original vertex. An image inspection processing part 200 performs an inspection by excluding the non-inspection area demarcated by the non-inspection region setting part 138 from an inspection object. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention provides a method and apparatus for reading information such as characters, figures, and images written on a medium such as paper, and comparing the read captured image with a reference image to perform image inspection, and an electronic computer ( A program for performing image inspection using a computer.

  2. Description of the Related Art An image reading apparatus for reading an image of a document, a so-called scanner apparatus, is used in various fields. For example, it may be used alone, or may be incorporated in a copying apparatus or the like. Alternatively, it may be used in an image inspection apparatus that inspects an image defect or the like by comparing a read image with a reference image for inspection.

  In the image inspection, a method of detecting an image defect by comparing a captured image obtained by taking a printed material to be inspected into an image memory by an imaging camera or the like with a preset reference image is adopted. In addition, the image defect detection method performs a subtraction for each pixel between the captured image and the reference image, calculates a normalized cross-correlation using the divided reference image as a template, and calculates a portion where the correlation coefficient falls below the reference value. There is a method of detecting that an image defect has occurred.

  For example, Patent Literature 1 discloses a method of automatically adjusting image quality based on an image difference between an original image and a copied image, assuming that a copying apparatus incorporates an image inspection mechanism. In this method, standard document data for image quality adjustment is stored, the standard document data is output at the time of image quality adjustment, the output paper is digitized by an image reading device, and the density of a specified comparison point in two image data is determined. Take the difference of The image quality is maintained by adjusting the parameters of each unit of the copying apparatus based on the difference value.

JP 08-190630 A

  On the other hand, as an image reading apparatus, for example, a flatbed image scanner in which a document is placed on a platen glass serving as a document placing table and a line sensor scans below the document, and a user scans the image sensor by hand. There is a handy image scanner that reads a document, or an indirect image scanner (also referred to as a non-contact type image scanner) that indirectly reads a document with a certain distance between the document and an imaging device (sensor). . The indirect image scanner has the advantage that the size and flatness of the object to be read are less restricted.

  As an example of the indirect image scanner, for example, Patent Document 1 proposes a method in which a print image is read from an obliquely upper side by an imaging device such as an area sensor and converted into an image viewed from the front. In the method of photographing obliquely from above, there is a problem that the image is distorted into a substantially trapezoidal shape (referred to as geometric distortion of the image), or a difference in the degree of blur of the image due to a shift in focus between the front and the back of the document, that is, blur. There is a problem that the characteristics depend on the place and the image quality is deteriorated.

  For this reason, in the invention described in Patent Literature 2, in order to correct the location dependency (geometric distortion) of the geometric shape and the location dependency of the blur characteristic in the captured image, the photographing camera is changed from the target pixel to be corrected. To obtain a high-quality captured image by performing a predetermined calculation based on the measurement result and controlling a correction coefficient for correcting geometric distortion and blur according to the distance value. Like that.

JP-A-2000-22869

  In Japanese Patent Application Laid-Open No. H11-163, original image data before raster development of an output product is obtained, image processing is performed in accordance with the characteristics of the image reading apparatus to obtain reference image data, and an image is inspected by comparison with the output product. An approach has been proposed. It has also been proposed to make a comparison in consideration of a positional deviation in an image reading apparatus. In this method, since image processing is performed by obtaining image data before raster development, accurate data can be obtained as reference data.

JP 2000-123176 A

  However, when an output document having no defect is read out of the imaging range and read, when compared with the captured image and the reference image, the accuracy of defect detection is reduced due to the effect of the protruding portion. For example, in a template including a protruding portion, the correlation coefficient is lower than the reference value, and thus the protruding portion is erroneously detected as a defect.

  The present invention has been made in view of the above circumstances, and it is an object of the present invention to provide an image inspection method capable of accurately inspecting a captured image even when an output document is read out of an imaging range. I do. Another object of the present invention is to provide an image inspection apparatus that implements the image inspection method of the present invention, and a program suitable for realizing the image inspection apparatus as software using an electronic computer.

  That is, the image inspection method according to the present invention is an image inspection method for inspecting the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection, and A line segment indicating the edge of the document is obtained, a positional relationship between the obtained line segment and a boundary line indicating the imaging range is specified, and the document portion protruding from the imaging range is determined based on the specified positional relationship. The inspection area is set, and the non-inspection area is excluded from the inspection target, and the image of the original is inspected by comparing the reference image with the image of the original in the captured image.

  For example, if the document has a rectangular shape such as a quadrangle, a straight-line formula indicating the edge of the square-shaped document is determined, and the coordinates of the intersection of the determined straight-line formula and the boundary line indicating the imaging range are specified. Thus, the positional relationship between the document margin and the boundary line indicating the imaging range is specified.

  An image inspection apparatus according to the present invention is an image inspection apparatus that inspects the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection. An imaging unit that acquires a captured image; a line segment calculation unit that determines a line segment indicating an edge of a document in the captured image acquired by the imaging unit; and a line that is determined by the line segment calculation unit and a boundary that indicates an imaging range. A positional relationship identification processing unit that identifies the positional relationship with the line, and a non-inspection area setting unit that sets a document portion that protrudes from the imaging range based on the positional relationship identified by the positional relationship identification processing unit as a non-inspection area. An image comparison unit that compares the reference image and the image of the original part in the captured image while excluding the non-inspection area set by the non-inspection area setting unit from the inspection target; and a document based on the comparison result by the image comparison unit. The partial image was assumed and an image inspecting unit configured to inspect.

  The invention set forth in the dependent claims defines further advantageous specific examples of the image inspection apparatus according to the present invention. Further, the program according to the present invention is suitable for implementing the image inspection method and apparatus according to the present invention by software using an electronic computer (computer). The program may be provided by being stored in a computer-readable storage medium, or may be distributed via a wired or wireless communication unit.

  For example, when handling a document exhibiting a square shape such as a quadrangle, the line segment calculation unit is a straight-line expression calculation unit that obtains a straight-line expression indicating the edge of the document in the captured image obtained by the imaging unit, The positional relationship identification processing unit is a boundary part identification processing unit that identifies the coordinates of the intersection between the straight line formula obtained by the linear expression calculation unit and the boundary line indicating the imaging range, and the non-inspection area setting unit is a boundary part identification processing unit. What is necessary is just to set the document part which protrudes from the imaging range based on the specified intersection coordinates as a non-inspection area.

  According to the configuration of the present invention, a line segment indicating an edge of a document in a captured image is obtained, and a positional relationship (for example, intersection coordinates) between the line segment and a boundary line indicating an image capturing range is specified, thereby obtaining an image capturing range. The protruding portion of the original is set as a non-inspection area. Then, the image of the document portion is inspected by comparing the reference image with the image of the document portion in the captured image while excluding the non-inspection area from the inspection target.

  This makes it possible to inspect the output document without erroneous detection of a defect even if the output document is read out of the imaging range by excluding the protruding portion from the inspection target. And the inspection accuracy is improved.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a schematic diagram showing an image inspection apparatus provided with an embodiment of the image processing apparatus according to the present invention. Here, FIG. 1A is a diagram illustrating an installation state of a camera head, and FIG. 1B is a diagram illustrating an example of a print engine.

  As shown in FIG. 1A, an image inspection device 5 includes an image reading unit 100, which is an example of an image reading device, and an image inspection processing unit that inspects an image read by the image reading unit 100 for a defect. 200, a data storage unit 300 that stores an image read by the image reading unit 100, a reference image used for inspection by the image inspection processing unit 200, or intermediate data during data processing, and an image read by the image reading unit 100. A printer unit 500 for outputting to a predetermined recording medium is provided.

  The printer section 500 has a discharge port 582 and a paper discharge tray 584, and a print engine is arranged in a housing. The paper discharge tray 584 having a function as a document mounting table is colored so that the document (printed paper) P and the background (here, the color of the paper discharge tray) can be distinguished.

  The image reading unit 100 has a camera head 102 that is an example of an imaging unit that reads an image of a document P. The camera head 102 includes a photographing lens 104, a CCD image sensor 106, and a read signal processing unit 108. The camera head 102 is supported by a support member (not shown) such that the direction of the photographing lens 104 with respect to P discharged onto the paper discharge tray 584 is adjustable. The camera is installed so as to be able to capture images from diagonally above to obtain captured images.

  The printer section 500 is operable with various functions of the image reading section 100, and includes a raster output scan (ROS) -based print engine 570 for operating the image inspection apparatus 5 as a digital printing system.

  The print engine 570 records an image on a predetermined recording medium using an electrophotographic process. The print engine 570 functions as a printing device (printer) that forms a visible image on a predetermined recording medium based on print data input from a client terminal.

  That is, as shown in FIG. 1B, the print engine 570 has the function of an interface (IOT controller) with the image reading unit 100, and also prints image data D 10 output from the image reading unit 100. A print output processing unit 572 that performs a predetermined process for print output, a laser light source 574 that emits a light beam, a laser drive unit 576 that controls or modulates the laser light source 574 according to data output from the print output processing unit 572, A polygon mirror (rotating polygon mirror) 578 for reflecting the light beam emitted from the laser light source 574 toward the photosensitive member 579.

  The print output processing unit 572 generates and renders (decompresses into raster data) the image data D10 according to a known technique, generating data representing a plurality, preferably at least three, of separated colors. For example, mapping is performed to at least three (preferably four) from the YCrCb color system represented by the data D10, for example, to the CMY color system or the CMYK color system, and color-separated raster data for print output is generated.

  The print output processing unit 572 performs under color removal (UCR) for reducing the CMY components of the color image in such rasterization processing, or a gray component for partially replacing the reduced CMY components with the K component. Exchange (GCR). Further, the print output processing unit 572 may perform linearization of color separation or similar processing in order to adjust a toner image of an output image created in response to output data (such as CMYK).

  With this configuration, the print engine 570 reflects the light beam generated by the laser light source 574 on a plurality of surfaces on the polygon mirror 578 to expose the photosensitive member 579, and forms a latent image on the photosensitive member 579 by scan scanning. Form. When the latent image is formed, the image is developed according to any of a number of methods known in the art, and the color image read by image reading unit 100 is output as a visible image. When a color image is output as a visible image, data representing the image includes at least three (preferably four) color separation data (for example, C, M, Y, K, etc.), and each color is separated. Or in luminance-chrominance format.

<< 1st Embodiment >>
FIG. 2 is a block diagram showing a detailed example of the first embodiment of the image inspection device 5 shown in FIG. The image inspection apparatus 5 according to the first embodiment performs a perspective transformation process (geometric transformation process) on a captured image in which the vertices of a substantially rectangular original are out of the imaging range, and also performs a rectangular transformation of the perspective transformation process. Based on the image, position information (hereinafter, also referred to as non-inspection area information) for specifying an area (hereinafter, also referred to as non-inspection area) for protruding from the imaging range of the document portion is obtained. When performing an image inspection by comparing with a reference image, the image inspection is performed by referring to the previously obtained non-inspection area information and excluding the non-inspection area of the document from the inspection target. This will be specifically described below.

  The image inspection device 5 according to the first embodiment includes an image reading unit 100 that captures an image and obtains a captured image, an image inspection processing unit 200, a data storage unit 300, and a control unit 320 that controls each unit of the image inspection device 5. And a RAM (Random Access Memory) 322 for temporarily storing data being processed.

  The data storage unit 300 stores a captured image acquired by the camera head 102, a processed image processed by the captured image processing unit 110, various intermediate calculation results used for processing in image inspection, and the like.

  The image reading unit 100 includes a captured image processing unit 110 that performs a predetermined process on a captured image acquired by the camera head 102. The captured image processing unit 110 corrects shading by the shading correction unit 112 and the shading correction unit 112 that corrects uneven illuminance (light amount distribution characteristics of the optical system) and variation in sensor pixel characteristics of the captured image read by the camera head 102. An optical distortion correction unit 114 that corrects distortion (curvature) aberration mainly caused by the photographing lens 104 in the captured image, and a perspective that transforms the captured image corrected by the optical distortion correction unit 114 into a rectangular shape (perspective transformation). A conversion unit (geometric conversion unit) 116.

  The shading correction performed by the shading correction unit 112 is a process for correcting unevenness in illuminance of a photographing area, variation in characteristics of each pixel of the CCD image sensor 106, and the like. For example, the camera head 102 photographs a blank sheet in advance and stores it in the data storage unit 300 as a white reference DW (i, j). i and j represent pixel positions. Next, the shading correction unit 112 corrects the captured image D (i, j) based on Expression (1).

  Here, “n” is the bit resolution after correction. If the resolution is 8 bits, n = 8, and the value of the gradation is “0 to 255”. Here, the method of holding the white reference data for all pixels has been described. For example, in a simplified manner, the peak value of the entire captured image is set as the white reference DW, or the peak value DW (j) of each line is used. May be applied as a white reference.

  The optical distortion correction unit 114 corrects the aberration caused by the photographing lens 104 as follows. For example, assuming that the aberration at the incident angle θ to the photographing lens 104 is d, the distance from the photographing lens 104 to the image plane of the CCD image sensor 106 is c, and the distance from the optical axis of the image forming position on the image plane is r. , And aberration d are expressed by the following equation (2). Therefore, the optical distortion correction unit 114 corrects distortion based on this characteristic. Since the aberration d is generally proportional to the cube of r, the distortion can be corrected by obtaining a proportional constant based on the lens characteristics.

  In the present embodiment, since the camera head 102 captures the document P from obliquely above, the rectangular document P has a trapezoidal shape with respect to the captured image area on the captured image data. The perspective conversion unit 116 converts the captured image so as to match the original rectangular shape.

  In addition, the captured image processing unit 110 includes a non-inspection area specifying processing unit 120 that specifies an area of the document P in the captured image and also specifies an area (hereinafter, also referred to as a non-inspection area) that protrudes from the imaging range of the document part. . The non-inspection area identification processing unit 120 identifies a non-inspection area that is outside the imaging range of the document portion, and stores information indicating the non-inspection area (hereinafter, referred to as non-inspection area information) in the data storage unit 300.

  Each time the camera head 102 captures an image, the non-inspection area identification processing unit 120 extracts a document area from the captured image. Then, by referring to the document area information extracted each time the photographing is performed, the non-inspection area protruding from the imaging range is specified.

  It is considered that the place and direction in which the document P is discharged to the paper discharge tray 584 is fixed to some extent, and it is possible to fix the inspection target range. Or it is ejected at an angle. When the amount is large, the original portion is read out of the imaging range. In this case, the protruding portion may be erroneously detected as a defect. Therefore, as in the present embodiment, the actual portion of the document P and the non-inspection area are specified each time shooting is performed, and the specified non-inspection area is excluded from the target of the image inspection processing. Will be able to

  As a mechanism for performing error correction for reducing an error between a captured image read by the camera head 102 and a reference image for inspection, for example, as proposed by the present applicant in Japanese Patent Application No. 2001-334163. A correction coefficient calculation unit and an image correction processing unit may be provided. For details of the correction coefficient calculation unit and the image correction processing unit, refer to Japanese Patent Application No. 2001-334163.

  For example, the image correction processing unit performs an enhancement process (sharpness process) on a captured image acquired by the camera head 102 by applying an edge enhancement filter or the like generally used in image processing, This is an example of a noise reduction unit that reduces a noise component included in the captured image by applying a smoothing filter or the like to remove moire or smoothing halftone data to reduce a noise component included in a captured image. (Smoothing process).

  The correction coefficient calculation unit calculates a blur characteristic correction coefficient for correcting the location dependency of the blur characteristic due to the tilt scan of the captured image acquired by the camera head. For example, a correction coefficient for each line of the emphasis processing or the noise removal processing is calculated from the trapezoidal document area, and the calculated correction coefficient is set in the image correction processing unit.

  The image correction processing unit adjusts the parameters of each filter with the correction coefficient calculated by the correction coefficient calculation unit when performing the enhancement processing and the smoothing processing. In this case, it is more preferable that both the sharpness and the degree of noise removal are changed in line units, that is, it is preferable that the enhancement process and the noise removal process are simultaneously performed in line units.

  The image inspection processing unit 200 converts the reference image (original image data) stored in the data storage unit 300, which is a reference for image inspection, into a camera similarly to the configuration proposed by the present applicant in Japanese Patent Application No. 2001-334163. A reference image processing unit 210 that converts the resolution in the same manner as the captured image acquired by the head 102 and performs a blurring process (smoothing process) to reduce errors when comparing the two images; An image inspection unit 230 that inspects the quality of the captured image based on the reference image output from the processing unit 210 is provided.

  The reference image processing unit 210 performs a resolution conversion process on the reference image read from the data storage unit 300 using a linear interpolation method or the like, and a high-frequency conversion process on the reference image subjected to the resolution conversion process. A shading processing unit 216 for performing shading processing for reducing components, and a non-inspection area deletion for excluding, from the inspection target area, an area corresponding to a part that is outside the imaging range from the reference image subjected to the shading processing. A processing unit 218.

  The blurring processing unit 216 performs a gaussian variance σ on the reference image subjected to the resolution conversion by the resolution conversion unit 214 so as to have the same blurring degree of the captured image (that is, the Gaussian filter). (Using) to adjust (generally reduce) the sharpness of the reference image for inspection. In short, the blur caused by the optical system of the camera head 102 in the captured image data is reflected. At this time, the correction coefficient of the Gaussian filter processing is calculated with reference to the trapezoidal shape of the document area immediately after the imaging, and the reference image is blurred (Gaussian filter processing) based on this. Note that the Gaussian filter processing to which the variance σ is applied is a known technique, as shown in, for example, Japanese Patent Application Laid-Open No. 5-225388, and therefore a detailed description thereof is omitted.

  3A and 3B are diagrams showing the relationship between the captured image and the reference image, wherein FIG. 3A is a diagram illustrating a shooting state of an image read by the camera head 102, FIG. 3B is a diagram illustrating the captured image, and FIG. FIG. 7C is a diagram illustrating an image when converted into a shape.

  In the present embodiment, since the camera head 102 captures a rectangular document at an arbitrary tilt angle, the document portion of the captured image captured by the camera head 102 has a trapezoidal image as shown in FIG. It is transformed and acquired. The image data representing the trapezoidal image is temporarily stored in the data storage unit 300.

  Also, as shown in FIG. 3A, the document P is photographed by tilt scan with the camera head 102 focused on the front edge (the thick line in the figure) of the document P. In this case, in the trapezoid original area (see FIG. 3B) of the captured image data, the lower bottom (lower side) of the original P is focused, so that the focus gradually moves toward the upper bottom (upper side). It will be off. That is, the blur amount of the image differs depending on the pixel position.

  In order to correct this effect, the perspective transformation unit 116 matches the shape of the trapezoidal image with the original rectangular shape. For this purpose, for example, there is a method of applying a conversion formula used when returning a trapezoidal shape of a captured image as shown in FIG. 3B to a rectangular shape as shown in FIG. 3C.

  For example, as a perspective conversion process for returning a trapezoidal shape of a captured image to a rectangular shape, a method of performing conversion based on a distance from a document P to a camera, a camera height h, a pan angle, and a tilt angle is known. This is obtained by solving a matrix operation using a homogeneous coordinate expression to derive a conversion formula. The conversion formula is represented by the following formulas (3) and (4).

Here, u and v are the coordinates after conversion, x and y are the coordinates of the source image, h is the camera height from the document surface, α (= tan ⁻¹ (d / h)) is the tilt angle of the camera, β Is the pan angle of the camera. By substituting each pixel of the captured image into the perspective transformation formula from the camera height h, the tilt angle α, and the pan angle β measured in advance, the trapezoid document area can be returned to a rectangle, that is, perspective transformation processing can be performed.

  FIG. 4 is a block diagram illustrating a configuration example of the non-inspection area specifying processing unit 120 according to the first embodiment. As shown in the drawing, the non-inspection area specifying processing unit 120 includes an edge extracting unit 122 that extracts an edge portion from a captured image, and an edge valid unit detecting unit 124 that selects an effective component in the edge portion extracted by the edge extracting unit 122. And an edge-effective-part linear-expression calculating unit that is an example of a line-segment calculating unit that calculates a linear expression for determining a document part in a captured image with reference to the valid edge part specified by the edge-effective-part detecting unit 124 126.

  In addition, the non-inspection area specifying processing unit 120 refers to the straight line formula calculated by the edge effective part straight line calculation unit 126 to specify an area (corresponding to a non-inspection area) that is outside the imaging range of the document. An inspection area specifying unit 130 is provided. The non-inspection area specifying unit 130 is a document vertex setting process which is an example of a positional relationship specifying processing unit that specifies the vertex coordinates of the document in the captured image with reference to the linear equation calculated by the edge effective unit linear equation calculating unit 126. And a protruding place determination processing unit 134 that determines whether the vertex coordinates set by the document vertex setting processing unit 132 are out of the imaging range.

  Further, the non-inspection area specifying unit 130 includes a boundary part specifying processing unit 136 that specifies a boundary part of the document vertex whose vertex coordinates are determined to protrude from the imaging range by the protruding place determination processing unit 134 with the imaging range. The non-inspection area is determined based on the coordinate information indicating the boundary part specified by the boundary part specifying processing unit 136 and the coordinate information of the document vertex whose vertex coordinates are determined to protrude from the imaging range by the protruding place determination processing unit 134. And a non-inspection area setting unit 138 that sets

  The boundary part specifying processing unit 136 is calculated by the edge valid part linear expression calculating unit 126 that defines the document vertex of the document vertex whose vertex coordinates are determined to be outside the imaging range by the protruding place determination processing unit 134. The coordinates of the intersection formed by the two straight-line equations and the straight-line equation indicating the imaging range are obtained as information (hereinafter, referred to as boundary coordinates) for specifying the boundary portion. The non-inspection area setting unit 138 sets, as a non-inspection area, an area surrounded by the vertex coordinates of the protruding portion and the boundary coordinates obtained corresponding to the vertex coordinates.

  FIG. 5 is a diagram illustrating a state in which a document portion to be processed according to the present embodiment is imaged out of the imaging range. In each figure, the coordinates of the top, bottom, left and right vertices (xαβ, yαβ) of the document are defined as follows. That is, the coordinates of the upper left vertex of the document are (xtl, ytl), the coordinates of the lower left vertex of the document are (xbl, ybl), the coordinates of the upper right vertex of the document are (xtr, ytr), and the coordinates of the lower right vertex of the document are (xbr, ybr). And

  As shown in FIG. 3, x and y correspond to the scanning direction of the captured image. The symbol “α” means upper side (top) when “t” and lower side (bottom) when “b”. The symbol “β” means “left” when “l” and right side when “r”. The meaning of this code is the same for other codes described later. In the drawing, “M → N” indicates that the vertex M of the document protrudes in the N direction of the imaging range.

  The shape of the portion of the document that protrudes from the imaging range changes depending on the amount of deviation and the inclination of the document with respect to the imaging range. Further, when a part of the document protrudes from the imaging range, at least one of the four vertices of the document protrudes from the imaging range. However, if three vertices are within the imaging range, all four sides are at least partially within the imaging range. In this embodiment, as described above, one of the four vertices of the rectangular document protrudes from the imaging range, and the remaining three vertices protrude from the imaging range. In the illustrated example, FIGS. 5A and 5B show a case where the original is displaced in a state where the original is moved in parallel with no inclination with respect to the imaging range. FIG. 5C to FIG. 5J show a state in which the image is protruded with an inclination.

  The illustrated example is an example, and the relationship between the vertex α and the boundary of the imaging range may take a state other than the illustrated state depending on the inclination state or the like. Even in this case, one vertex is out of the imaging range and the remaining three vertices are imaged. Everything within the range is the processing range of the present embodiment. On the other hand, a protruding state in which two or more vertices are out of the imaging range is not a processing target of the present embodiment. Although the present embodiment does not correspond to all possible modes of the protruding state, if the protruding amount is not so large, generally, one vertex is outside the imaging range and the remaining three vertices are within the imaging range. I don't think this is a problem.

  The non-inspection area specification processing unit 120 appropriately specifies the document portion that protrudes from the imaging range according to the state of the document illustrated in FIGS. 5A to 5J, and determines the protruding portion. Set as an inspection area.

  FIG. 6 is a flowchart illustrating an outline of a processing procedure in the image inspection apparatus 5 according to the first embodiment. In the image inspection apparatus 5 of the first embodiment, when the printer unit 500 is in a standby state, the camera head 102 periodically captures an image on the paper output tray 584 and outputs the image according to an instruction from the control unit 320. .

  The control unit 320 compares the image of the paper output tray 584 stored in the data storage unit 300, which is an example of the reference image, with the captured image obtained by the camera head 102, and the document is on the paper output tray 584. It is determined whether or not the document is on the discharge tray 584, the image inspection possible flag is turned off, and the standby state is continued. On the other hand, if no document is on the discharge tray 584, the image inspection is possible. The flag is turned on and the standby state is continued (S100).

  Then, when receiving a print job in the standby state (S102), the control unit 320 determines a flag indicating that image inspection is possible (S104). Then, when the image inspection possible flag is off (S104-No), the control unit 320 causes the print engine 570 to perform normal print processing (S106), and returns to the standby state (S180). On the other hand, when the image inspection possible flag is on (S104-Yes), the control unit 320 executes the image inspection on the Nth print job (first, naturally, the first print job) as follows.

  That is, the control unit 320 first causes the print engine 570 to start print processing of the N-th sheet (first, naturally, the first sheet) (S108), and outputs the developed image data to the print output processing unit 572 of the print engine 570. And stores it in the data storage unit 300. When the rendering processing of the last image data is completed, a signal of completion of processing of the first sheet from the print output processing unit 572 is waited (S130). When a completion signal comes, a shooting start trigger signal is sent to the camera head 102 after a prescribed time. Is sent (S132).

  The camera head 102 releases the shutter (photographs) in synchronization with the trigger signal, but at this time immediately after the document P (print output paper) is output onto the paper discharge tray 584. The camera head 102 stores a trapezoidal captured image obtained by photographing a rectangular document in the data storage unit 300.

  When the print job is a plurality of sheets (N sheets), the control unit 320 proceeds with the remaining print processing in parallel with the processing (S130 to S162) described later until the processing for all the plurality of sheets (N sheets) is completed. (S110). Although these processes are shown as parallel processes in the figure, they can be executed as time-division sequential processes. When the plurality of sheets (N sheets) are all completed, the control unit 320 returns to the standby state (S112). In this manner, all captured images captured by the camera head 102 are converted into digital signals and stored in the data storage unit 300.

  The captured image processing unit 110 performs pre-processing on the captured image captured by the camera head 102. That is, the shading correction unit 112 reads the captured image captured by the camera head 102 from the data storage unit 300, and performs shading correction for correcting uneven illuminance and variation in sensor pixel characteristics (S134). Further, the optical distortion correction unit 114 performs optical distortion correction for correcting the distortion and the like of the photographing lens 104 on the image on which the shading correction has been performed, and stores the processed image in the data storage unit 300 (S136). ).

  Next, the control unit 320 determines whether or not to perform perspective transformation (perspective transformation) on the captured image on which shading correction and optical distortion correction have been performed by the captured image processing unit 110 (S140). When performing the perspective transformation process on the captured image (S140-Yes), the control unit 320 instructs the perspective transformation unit 116 provided on the captured image processing unit 110 side to perform the perspective transformation process. The perspective transformation unit 116 that has received this instruction performs perspective transformation processing on the trapezoidal document area into a rectangle based on the above-described transformation equations (3) and (4) for the captured image that has been subjected to shading correction. Is stored in the data storage unit 300 (S142).

  After that, the control unit 320 instructs the non-inspection area identification processing unit 120 to identify the document portion that is out of the imaging range. The non-inspection area identification processing unit 120 that has received the instruction executes the non-inspection area identification processing according to a procedure described later, and stores information indicating the area of the protruding portion (non-inspection area) in the data storage unit 300 ( S144).

  Next, the resolution conversion unit 214 of the reference image processing unit 210 adjusts the resolution of the reference image so that the size of the rectangular reference image is equal to the size of the image of the rectangular document portion in the captured image to be inspected. Is converted (S152).

  Next, the blurring processing unit 216 extracts the non-inspection area specifying unit 130 to reflect the blurring caused by the optical system of the camera head 102 in the captured image data in the resolution-converted reference image. With reference to the obtained vertex coordinates A, B, C, and D, the value of the correction coefficient corresponding to the trapezoidal shape of the image used in the Gaussian filter process is calculated, and based on this, the blurring process (Gaussian process) is performed on the reference image. Filter processing) (S154).

  Here, the blurring process is performed on the reference image because the reference image has an ideal density characteristic, and the blurring process equivalent to the blurring characteristic of the captured image is performed. This is because the blurring process can be performed in accordance with the blurring distribution, and an error in the difference processing described below can be minimized.

  Next, the non-inspection area deletion processing unit 218 reads out the non-inspection area information specified by the non-inspection area specifying unit 130 from the data storage unit 300, and performs the shading processing by the shading processing unit 216 on the reference image. The non-inspection area is set for the reference image by invalidating the data of the non-inspection area in (S156). The non-inspection area setting unit 138 stores the processed reference image in the data storage unit 300.

  Next, the alignment processing unit 232 performs processing on the image reading unit 100 with the reference image that has been subjected to the blurring processing by the blurring processing unit 216 of the reference image processing unit 210 and the non-inspection area portion has been invalidated. The registration process of the pre-processed captured image is started (S158). For example, the alignment processing unit 232 sequentially calculates the normalized cross-correlation coefficient while scanning the captured image using the reference image as a template, and regards a position having the highest correlation coefficient as a portion where the position is matched, so-called Positioning is performed by template matching.

  Next, the defect abnormality detection processing unit 234 transmits the inspection image (trapezoidal shape) that has been pre-processed by the captured image processing unit 110 and stored in the data storage unit 300 to the reference image processing unit 210. Then, a difference process is performed to obtain a difference from the reference image (with a trapezoidal shape) subjected to the perspective transformation process or the blurring process (compare the two) (S160). Then, the defect abnormality detection processing unit 234 detects the presence or absence of a defect or abnormality in the captured image based on the difference result obtained by the difference processing (S162). The alignment processing unit 232 and the defect abnormality detection processing unit 234 perform the above processing on the entire surface of the reference image, and after completion, return to the standby state again.

  For example, when the normalized cross-correlation coefficient at a position where the position is matched is lower than the determination index value, the defect abnormality detection processing unit 234 performs a defect abnormality detection process on the assumption that an image defect exists at that portion. Thus, for example, if there is a defect pattern such as a black spot in a print output, that is, a captured image, the normalized cross-correlation coefficient clearly decreases at a location including the defect, and a black spot remains in the difference result.

  Therefore, for example, by setting an appropriate density threshold and binarizing the difference result, a black point can be specified, and the presence or absence of an image defect can be easily determined. The present invention is not limited to the detection of local image defects such as black spots, and can be applied to the detection of image quality defects such as overall density unevenness and density shift of a captured image.

  When the defect abnormality detection processing unit 234 detects an image defect or a poor image quality, the image inspection device 5 issues a warning on a user interface (not shown) such as a predetermined display device or an audio signal generator (for example, a speaker). Using a print management software or the like, a warning is issued to a client terminal (for example, a personal computer) trying to print through a communication network, and further a notification is sent to a printer maintenance company through a communication network via a remote maintenance system or the like.

  FIG. 7 is a diagram illustrating an example of an image handled in the first embodiment. Here, FIG. 7A is a diagram showing a state where a document is properly placed within an imaging range, and FIG. 7B is a result of image pickup by the camera head 102 in the state of FIG. 7A. FIG. 7C is a diagram showing a result of perspective transformation performed on the image of FIG. 7B by the perspective transformation unit 116.

  The original portion in the image data before the perspective transformation processing captured by the camera head 102 has a trapezoidal shape as shown in FIG. When the perspective transformation is performed on this image by the perspective transformation unit 116, the original document returns to a rectangular shape as shown in FIG. 7C, but the outline of the imaging range has a trapezoidal shape. In the subsequent signal processing, a desired range (the thick line portion in the figure) including the document portion in the trapezoidal imaging range is cut out into a rectangular shape to obtain a captured image to be processed.

  Note that, as shown in FIG. 7C, the straight line formulas of the outline defining the trapezoidal imaging range are defined as the following expressions (5) to (8).

  FIG. 8 is a diagram illustrating another example of an image handled in the first embodiment. Here, FIG. 8A shows a result of imaging by the camera head 102 in a state where the original is placed without rotating with respect to the imaging range and a part of the original protrudes from the imaging range. FIG. 8B is a diagram showing a result of perspective transformation performed on the image of FIG. 8A by the perspective transformation unit 116.

  The original portion in the image data before the perspective transformation processing captured by the camera head 102 has a trapezoidal shape as shown in FIG. When a perspective transformation is performed on this image by the perspective transformation unit 116, the original document returns to a rectangular shape as shown in FIG. 8B, but the portion corresponding to the portion that protrudes from the imaging range is an invalid portion. Appears as. Therefore, even when a desired range (the thick line portion in the figure) including the document portion in the imaging range is cut out into a rectangular shape to obtain a captured image to be processed, the invalid data is included in the image data.

  FIG. 9 is a diagram for explaining the processing of the edge extraction unit 122. The edge extraction unit 122 performs edge extraction on the captured image that has been perspectively transformed by the perspective transformation unit 116. Edge extraction is performed by, for example, searching from the upper, left, lower, and right directions using a prewitt operator. For example, each operator on the upper side, left side, lower side, and right side expressed by the following equation (9) is applied. Thereby, the edge extraction images shown in FIGS. 9A to 9D are obtained.

  Next, a sum is obtained for each pixel of the edge extraction images shown in FIGS. 9A to 9D to obtain a combined edge extraction image shown in FIG. 9E. The edge-effective-portion detection unit 124 performs the edge-effective-portion detection process based on the edge-extracted image shown in FIG. 9E, thereby obtaining an edge-extracted image shown in FIG. 9F. In FIG. 9F, a portion indicated by four straight lines (thick lines in the drawing) is an edge effective portion. In FIG. 9 (F), the gradation value of the protruding portion is indicated by zero (black).

  Note that, for the prewitt operator, edges are extracted by the upper, left, lower, and right operators shown in the following equation (10), and the combined edge extracted image shown in FIG. 9E is further added. Thus, it is possible to obtain an edge extraction image in which edges are further emphasized.

  FIG. 10 is a flowchart illustrating an outline of a procedure of the non-inspection area specifying process of the non-inspection area specifying processing unit 120 according to the first embodiment. In the non-inspection area specifying process according to the first embodiment, after performing a geometric transformation process (perspective transformation process) on a captured image in which a part of a document is out of an imaging range, a non-inspection region is set and inspected. Things.

  First, the edge extraction unit 122 extracts an edge component from the captured image that has been perspectively transformed by the perspective transformation unit 116 (S200). More specifically, the captured image subjected to the perspective transformation by the perspective transformation unit 116 is subjected to edge extraction by applying a prewitt operator for each of up, down, left, and right directions. As a result, an edge-extracted image as shown in FIGS. 9A to 9D is obtained.

  Next, the edge valid part detecting unit 124 detects one edge valid part for each of the upper, lower, left, and right with reference to the edge information extracted for each of the up, down, left, and right directions. Specifically, the valid edge detection unit 124 first specifies valid edge portions (vertical valid portions) for each of the upper, lower, left, and right directions (S210). When one edge effective portion can be specified, it becomes a straight line indicating the edge of the document. If there are a plurality of specified edge valid parts, the edge valid part detection unit 124 detects a more probable edge valid part from the specified plurality of edge valid parts (S300).

  Next, in order to specify the vertex coordinates of the document based on the edge valid part detected by the edge valid part detection unit 124, the edge valid part linear equation calculation unit 126 first executes the edge effective part linear equation calculation processing. , The lower side, the left side, and the right side (S400).

  Next, the document vertex setting processing unit 132 specifies the vertex coordinates of the document in the captured image with reference to the linear equation calculated by the edge effective unit linear equation calculating unit 126 (S420). The protruding place determination processing unit 134 determines whether the vertex coordinates set by the document vertex setting processing unit 132 are out of the imaging range (S500).

  Next, the boundary part specifying processing unit 136 specifies a boundary part of the document vertex whose vertex coordinates are determined to protrude from the imaging range by the protruding place determination processing unit 134 and the non-inspection area setting unit. 138 defines a non-inspection area based on the coordinate information indicating the specified boundary portion and the coordinate information of the document vertex whose vertex coordinates are determined to protrude from the imaging range by the protruding place determination processing unit 134 ( S600). The non-inspection area setting unit 138 registers information indicating the defined non-inspection area in the data storage unit 300.

  FIG. 11 is a flowchart showing the edge extraction processing (S200 in FIG. 10) in the non-inspection area specifying processing of the first example.

  The edge extraction unit 122 focuses on the left side and searches from the left side from the left side (S210L), focuses on the right side and searches from the right side on the right side (S210R), focuses on the upper side and searches from the upper side on the upper side Edge extraction processing is performed in the order of edge extraction processing (S210T) and lower edge extraction processing (S210B) that searches from the lower side by focusing on the lower side.

  FIG. 12 is a flowchart illustrating a detailed example of the left side edge extraction processing (S210L in FIG. 11) in the non-inspection area specifying processing of the first example.

  First, the edge extraction 122 initially sets the operator “i” to “0” (S210), and detects edge coordinates xl (i) when the y coordinate is yl (i) (S222-Yes, S224). This means that (x, y) is scanned as (0, yl (i)), (1, yl (i)), (2, yl (i)), (3, yl (i)), ... First, the point where the gradation value is “255”, that is, the point where black changes to white first is detected as the edge coordinates (xl (i), yl (i)).

  Here, the yl (i) coordinates to be scanned are set every several pixels, that is, the scan width is set to several pixels. The extracted edge coordinates are (xl (0), yl (0)), (xl (1), yl (1)), (xl (2), yl (2)), (xl (3), yl ( 3)),..., (Xl (i), yl (i)),.

  Next, the edge valid part detecting unit 124 determines, for all the extracted edge coordinates, between adjacent edge coordinates (xl (i-1), yl (i-1)) and (xl (i), yl (i)). Is calculated as in the equation (11) (S226-Yes, S228).

  The calculation process of the inclination al (i-1) is repeated for the imaging range to be processed (the thick line portion in the figure) while incrementing "i" (S230, S222). The setting value 1 is set to define this range, and may be set to a value obtained by dividing the number of pixels in the imaging range in each of the scanning directions (the y direction in this example) by the number of pixels of the scanning width. The reason why the step 228 is passed when “i = 0” is that the inclination al (i−1) cannot be obtained unless there are two edge coordinates (S226-No).

  Next, the edge valid part detection unit 124 initially sets the operator "i" to "0" and the operator "leftedgecnt" to "0" (S240). Further, the edge valid part start flag sflag is initially set to “0” (S242). Then, the edge valid part detecting unit 124 further calculates, for the calculated gradients al (1), al (2), al (3),..., Al (i),. The difference sl (i) of (i) is calculated as in equation (12) (S246).

  Next, the edge valid part detecting unit 124 determines whether or not the calculated difference sl (i) is equal to or smaller than the set value 2 (S250). The inclination difference using the edge pixels detected at the portion corresponding to the document margin can be considered to be almost zero. On the other hand, the inclination difference obtained as a result of scanning the protruding portion is an inclination difference using the edge pixels of the character portion inside the document, and can be considered to be relatively large. Therefore, it is preferable to determine the value of the set value 2 so as to remove the influence of noise.

  When sl (i) is equal to or less than the set value 2 (S250-Yes), the edge-effective-portion detection unit 124 sets the edge-effective-portion start flag sflag to “1” (S254). Then, (xl (i), yl (i)) at this time is set as the left edge effective portion start point (xsl (leftedgecnt), ysl (leftedgecnt)) (S256).

  After step S256, the operator "i" is incremented by "1" and the process returns to step S244 (S270). After that, if the value of sl (i) is equal to or less than the set value 2, the sflag remains "1", and is detected as an edge valid part.

  On the other hand, when sl (i) is larger than the set value 2 (S250-No), the edge-effective-portion detection unit 124 determines whether the edge-effective-portion start flag sflag is “1” (S262). Only when the edge valid part start flag sflag is "1", the edge valid part start flag sflag is set to "0" to end the edge valid part (S262-Yes, S264).

  Further, the edge valid part detecting unit 124 sets (xl (i), yl (i)) at this time as the left side edge valid part end point (xel (leftedgecnt), yel (leftedgecnt)) (S266), and the operator “i”. "And" leftedgecnt "are each incremented by" 1 ", and the process returns to step S244 (S272).

  When the operator “i” exceeds the “set value 1-1” (S244-No), the edge valid part detection unit 124 sets the left edge valid part start point (xsl (leftedgecnt), ysl (leftedgecnt)) to the left edge. An effective edge portion is set between the effective portion end point (xel (leftedgecnt) and yel (leftedgecnt)) (S278).

  Thus, (xsl (0), ysl (0)) and (xel (0), yel (0)), (xsl (1), ysl (1)) and (xel (1), yel (1) )),..., (Xsl (leftedgecnt), ysl (leftedgecnt)) and (xel (leftedgecnt), yel (leftedgecnt)) are detected and set as an effective edge portion. The number of detected valid edges is leftedgecnt.

  The right side edge valid part detection processing S210R, the upper side edge valid part detection processing S210UT, and the lower side valid edge part detection processing detection processing S210B also switch the scan direction according to each direction, and follow the processing procedure of the left side edge valid part detection processing. Process. As a result, the right side edge valid part becomes (xsr (0), ysr (0)) and (xer (0), yr (0)), (xsr (1), ysr (1)) and (xer (1), .., (xsr (rightedgecnt), ysr (rightedgecnt)) and (xer (rightedgecnt), yer (rightedgecnt)) are detected only rightedgecnt.

  The upper edge valid part is (xst (0), yst (0)) and (xet (0), yet (0)), (xst (1), yst (1)) and (xet (1), yet (1)),..., (Xst (topedgecnt), ys (topedgecnt)) and (xet (topedgecnt), yett (topedgecnt)) are detected only by topedgecnt. The lower edge valid part is (xsb (0), ysb (0)) and (xeb (0), yeb (0)), (xsb (1), ysb (1)) and (xeb (1), yeb (1)),..., (Xsb (bottomedgecnt), ysb (bottomedgecnt)) and (xeb (bottomedgecnt), yeb (bottomedgecnt)) are detected only by bottomedgecnt.

  FIG. 13 is a diagram illustrating a case where the above-described left side edge extraction processing is applied to the image of FIG.

  As shown in the figure, if the left-hand protruding portion of the document is searched from the left side, the first edge effective portion may be detected in some cases. Further, within the imaging range on the left side, an edge effective portion (second) indicating the left side itself is detected.

  FIG. 14 is a flowchart illustrating the edge valid part detection processing. FIG. 15 is a table showing the determination conditions used in the edge effective portion detection processing.

  The valid edge detecting unit 124 temporarily detects the valid left edge (S310L), temporarily detects the valid right edge (S310R), temporarily detects the valid upper edge (S310T), and temporarily detects the valid lower edge (S310B). ) Is performed, and then it is determined whether or not to detect an effective edge portion based on the values of lefthamidashi, righthamidashi, tophamidashi, and bottomhamidashi (S322). The conditions are as described in FIG. 15. If the left side and the right side or the upper side and the lower side run off simultaneously, the process ends without performing the non-inspection setting process as the error process (S322-No, S334).

  When the condition is satisfied (S322-Yes, S332), the edge valid part detection unit 124 performs processing for each of the extracted number of edge valid parts in the order of left side → right side → upper side → lower side (S332 to S364). Specifically, when the left side edge number flag “leftfukusu” detected in the left side effective edge portion detection (S310L) is “0”, the process directly proceeds to the processing of the right side edge effective part number (S332L-No). If leftfukusu is “1”, a left edge valid part detection process (S334) is performed (S332-Yes).

  Similarly, when the right side edge number flag rightfukusu detected in the right side effective edge portion detection (S310R) is “0”, the process directly proceeds to the processing of the number of upper edge effective parts (S342-No). If rightfukusu is “1”, the right side edge valid part detection processing (S344) is performed (S342-Yes). Similarly, when the upper side edge number flag topfukusu detected in the upper side valid edge portion detection (S310T) is “0”, the process directly proceeds to the processing of the lower side effective edge number (S352-No). If topfukusu is “1”, upper edge valid part detection processing (S354) is performed (S352-Yes). Similarly, when the lower side edge number flag bottomfukusu detected in the lower side valid edge portion detection (S310B) is “0”, the edge valid portion detection processing ends (S362-No). If bottomfukusu is “1”, the lower edge valid part detection process (S364) is performed (S362-Yes).

  FIG. 16 is a flowchart illustrating a detailed example of the left side edge valid part temporary detection processing (S310L in FIG. 11) in the first example non-inspection area specification processing.

  If the number of leftedgecnt is 0, error processing is performed without performing the non-inspection processing, and the processing ends (S3102-Yes). Next, when the detected number of left edge valid parts leftedgecnt is one, it is set as a temporary left edge valid part. At this time, (xsl (0), ysl (0)) is set to (xedgesl, yedgesl) and (xel (0), yel (0)) is set to (xedgeel, yedgeel) (S3112-Yes), (S3112-Yes) The distance dleft between (xedgesl, yedgesl) and (xedgeel, yedgeel) is calculated (S3138). If the value of “delett” is between v1 and v2 (S3152-Yes), lefthamidashi is set to “0” and leftfukusu is set to “0” (S3154). If the value is not a value between v1 and v2 (S3152-No), Lefthamidashi is set to "1" and leftfukusu is set to "0" (S3156). Here, the values of v1 and v2 are set to values in which an error is added to the horizontal length when the output document is photographed at a predetermined position.

  If the detected number of left edge valid parts leftedgecnt is neither zero nor one (two or more), left edge valid part selection processing is performed (S3112-Yes, S3114). Next, when leftedgecnt 'is "0", error processing is performed without performing the non-inspection processing, and the processing ends (S3122-Yes). Next, when leftedgecnt ′ is “1”, (xsl ′ (0), ysl ′ (0)) is set to (xedgesl, yedgesl), and (xel ′ (0), yel ′ (0)) is set to (xedgeel, yedgeel) (S3132-Yes), and the distance dleft between (xedgesl, yedgesl) and (xedgeel, yedgeel) is calculated (S3138). If the value of “delett” is between v1 and v2 (S3152-Yes), lefthamidashi is set to “0” and leftfukusu is set to “0” (S3154). If the value is not a value between v1 and v2 (S3152-No), Lefthamidashi is set to "1" and leftfukusu is set to "0" (S3156).

  If leftedgecnt 'is neither zero nor one, i is set to 0, and (xsl' (i), ysl '(i)) is changed to (xedgesl, yedgesl) and (xel' (i) while incrementing. , Yel '(i)) is set to (xedgeel, yedgeel) (S3144), lefthamidashi is set to “1”, leftfukusu is set to “1”, and then the left-side edge portion temporary detection processing is ended.

  FIG. 17 is a flowchart illustrating details of the left side edge valid part detection processing. First, i is set to 0 and leftedgecnt 'is set to 0 (S3200). While incrementing i, if it is smaller than leftedgecnt '(S3202), the slope aleft-st-end between (xsl (i), ysl (i)) and ((xel (i), yel (i)) is calculated. (S3204) If the absolute value of the left-st-end is larger than the set value left-pre (3212-Yes), (xsl (i), ysl (i)) is changed to (xsl '(leftedgecnt), ysl'). In (leftedgecnt)), set to (xel (i), yel (i)) and set to (xel '(leftedgecnt), yel' (leftedgecnt)) (3214). Is smaller than the set value aleft-pre, that is, processing is performed so that the case where the paper is extremely rotated is not detected as one side.

  Similarly, the provisional detection processing of the right side edge valid part (S310R), the provisional detection processing of the upper side edge valid part (S310T), and the provisional detection processing of the lower side edge valid part (S310B) are performed, whereby the edge for each side is obtained. The valid part processing ends.

  First, the valid edge detection unit 124 tentatively sets a valid edge on the right side. As the provisional right side effective edge portion, for example, one right side edge valid portion is detected in step S210R (specifically, the process corresponding to the right side in S220 to S278), and the right side edge valid portion is We will explain in the case where it is required correctly. In this case, the coordinates (xsr, ysr) giving the right side edge valid part start point edgers (1) are (xsr (i), ysr (i)), and the coordinates (xer, xer, giving the right side edge valid part end point edgere (1)). yer) is (xer (i), yer (i)). At this time, if the entire right side is within the imaging range, the distance between the two coordinates (xsr (i), ysr (i)) and (xer (i), yer (i)) is the length of the right side of the document. It will be almost equivalent.

  FIG. 18 is a flowchart illustrating details of the left side edge valid part detection processing. If the number of left edge valid portions leftedgecount is “1” (the number of left edge valid portions is two or more), the edge valid portion detecting section 124 performs processing for selecting a formal edge valid portion. If rightfukusu of the right side edge is 0 (the right side edge effective part is one), the inclination ar between the right side edge coordinates (xedgesr, yedgesr) and (xedgeer, yedgeer) is calculated (S3302-Yes, S3304).

  If rightfukusu of the right side edge is not 0 (the right side edge valid portion is one), the process ends without performing the non-inspection portion setting process as an error (S3302-No, S3318).

  Next, the inclination ar between the right side edge coordinates (xedgesr, yedgesr) and (xedgeer, yedgeer) is calculated (S3302-Yes, S3304). After that, the edge valid part detecting unit 124 calculates the intercept br of the right side effective edge part with the y-axis of the linear equation as in Expression (14) (S3306).

  Next, the edge valid part detecting unit 124 calculates the length dt of the upper side (or the length db of the lower side) based on the length dr of the right side and the size ratio of the reference image (corresponding to the aspect ratio of the document size). It is estimated (S3307, S3308). Specifically, first, the distance dr of the right side edge effective portion is calculated as in equation (15) (S3307), and then the horizontal length of the document is calculated by dr and the ratio of the vertical pixels to the horizontal pixels of the reference image. It is calculated as in (16) (S3308).

  Next, dlr1 and dlr2 for performing the process of determining the distance calculated in step S3316 are calculated as in equations (17) and (18), respectively (S3309). K1 and K2 set here are determined based on the positional relationship between the original and the camera.

  Next, the edge valid part detecting unit 124 initially sets the operator “n” to “0” (S3310). Then, it is determined whether the operator “n” is equal to or less than the operator “leftedgecnt” (S3312). Initially, it is naturally “less than or equal to” (S3312-Yes), and the edge valid part detection unit 124 sets the start point (xedsl (n), yedsl (n)) and end point of the left edge valid part when the operator is “n”. The midpoint (xml (n), yml (n)) between (xedel (n), yedel (n)) is calculated as in equation (19) (S3314). When the operator “n” is equal to or more than the operator “leftedgecnt” without detecting the left edge valid portion (S3312-No), the process ends without performing the non-inspection portion setting process as an error (S3312-No). , S3318). Next, the distance dl-r between (xml (n), ymm (n)) and the effective edge portion on the right side is calculated as in Expression (20) (S3312-Yes, S3314 to S3316).

  Next, the edge valid part detecting unit 124 determines whether or not the distance dl-r between both sides obtained by the equation (20) is between dlr1 and dlr2 obtained in S3309 (S3322). In the case where the distance between both sides is dl-rdlr1 and dlr2 (S3322-Yes), the edge valid part detecting unit 124 sets (xedgesl (n), yedgesl (n)) and (xedgeel (n), yedgeel) at that time. The process ends with (n)) as the formal left edge valid part (xedgesl, yedgesl) and (xedgeel, yedgeel) (S3324).

  If the distance dl-r between both sides is not between dlr1 and dlr2 (S3322-No), the edge valid part detecting unit 124 adds "1" to the operator "n" and returns to S3012.

  Similarly to the processing on the left side, the coordinates giving the start point of the right side edge effective portion are (xedgesr, yedgesr), and the coordinates giving the end point of the right side edge effective portion are (xedgeer, yedgeer). The coordinates giving the upper edge valid part start point) are (xedgest, yedgest), and the coordinates giving the upper edge valid part end point are (xedgeet, yedgeet). The coordinates giving the starting point of the lower edge valid part are (xedgesb, yedgesb), and the coordinates giving the ending point of the lower edge valid part are (xedgeeb, yedgeeb).

  Thus, the portion shown in FIG. 9F can be specified as an edge effective portion.

  The right side edge valid part selection processing S344, the upper side edge valid part selection processing S354, and the lower side valid edge part selection processing detection processing S364 may be in accordance with the procedures in the flowcharts of FIGS.

  FIG. 20 is a flowchart illustrating a detailed example of the edge-effective-portion straight-line calculating process (S400 in FIG. 10) by the edge-effective-portion straight-line calculating unit 126 according to the first embodiment. FIG. 21 is a flowchart showing a detailed example of the document vertex setting process (S420 in FIG. 10) by the document vertex setting processing unit 132.

  First, when the effective edge detection section (S210T in FIG. 11) detects the effective edge portion, the effective edge portion linear expression calculating section 126 first determines (xedgest, yedgest) and (xedgeet, yt) is calculated by equation (21T) (S402T). Similarly, when an effective edge portion has been detected in the lower edge effective portion detection process (S210B in FIG. 11), a linear expression yb between (xedgesb, yedgesb) and (xedgeeb, yedgeeb) of the lower edge effective portion is expressed by the following equation. It is calculated by (21B) (S402B).

  In addition, when the effective edge portion has been detected in the left edge effective portion detection process (S210L in FIG. 11), the edge effective portion linear expression calculating unit 126 determines (xedgesl, yedgesl) and (xedgeel, yedgeel) is calculated by the equation (21L) (S402L). Further, when an edge valid part has been detected in the right side edge valid part detection processing (S210R in FIG. 11), a straight line equation yr between (xedgesr, yedgesr) and (xedgeer, yedgeer) of the right side edge valid part is expressed by the equation ( 21R) (S402R).

  Next, the document vertex setting processing unit 132 sets the coordinates of the intersection of the straight line yt on the upper side and the straight line yl on the left side as the upper right vertex (xtl, ytl) of the document (S420). Next, the document vertex setting processing unit 132 sets the coordinates of the intersection of the straight line yb on the lower side and the straight line yl on the left side to the upper right vertex (xbl, ybl) of the document (S430). Next, the document vertex setting processing unit 132 sets the coordinates of the intersection of the straight line yt on the upper side and the straight line yr on the left side as the upper right vertex (xtr, ytr) of the document (S440). Next, the document vertex setting processing unit 132 sets the coordinates of the intersection of the straight line yb on the upper side and the straight line yr on the left side to the vertex (xbr, ybr) on the upper right of the document (S450), and executes the edge effective part linear equation calculation process. finish.

  FIG. 22 is a flowchart illustrating a detailed example of the protruding place determination processing (S500 in FIG. 10) by the protruding place determination processing unit 134 according to the first embodiment. FIG. 35 shows an example of the protruding place determination processing.

  The protruding place determination processing unit 134 first determines whether the top left vertex (xtl, ytl) of the document protrudes to the left (S502). From FIG. 7C, the straight line on the left boundary after the perspective transformation can be expressed by Expression (22). Therefore, whether the upper left vertex (xtl, ytl) is located to the left of the straight line on the left boundary is determined by substituting (xtl, ytl) into equation (22) as shown in equation (23). It can be determined whether it is larger or smaller. If (xtl, ytl) is located on the left side of the boundary shown in Expression (22), the condition of Expression (23) is satisfied.

  Similarly, the protruding place determination processing unit 134 determines whether the top left vertex (xtl, ytl) of the document protrudes upward (S504, S510). From FIG. 7C, the straight line of the upper boundary after the perspective transformation can be expressed by Expression (24). Therefore, whether the upper left vertex (xtl, ytl) is positioned above the straight line of the upper boundary is determined by substituting (xtl, ytl) into equation (24), as shown in equation (25). It can be determined whether it is larger or smaller. If (xtl, ytl) is located above the boundary indicated by the equation (24), the condition of the equation (25) is satisfied.

  If the expression (23) is satisfied and the expression (25) is also satisfied (S502-Yes, S504-Yes), the protruding place determination processing unit 134 sets the upper left vertex (xtl, ytl) as shown in FIG. As shown in (2), it is located on the left side of the boundary shown by the equation (22) and above the boundary shown by the equation (22). In this case, since the length of any side of the document is not known, that is, there is no one that fits the entire side within the imaging range, the protruding place determination processing unit 134 performs an error process (S506). ).

  When equation (23) is satisfied and equation (25) is not satisfied (S502-Yes, S504-No), the upper left vertex (xtl, ytl) is calculated using equation (22) as shown in FIG. It is located on the left side of the boundary line shown and below the boundary line shown by the equation (24). In this case, the protruding place determination processing unit 134 determines that the document protrudes to the left side, and sets flagtl to “1” (S508).

  When Expression (23) is not satisfied and Expression (25) is satisfied (S502-No, S510-Yes), the upper left vertex (xtl, ytl) is calculated by Expression (22) as shown in FIG. It is located on the right side of the boundary line shown and above the boundary line shown by the equation (24). In this case, the protruding place determination processing unit 134 determines that the document protrudes upward, and sets flagtl to “2” (S512). In addition, when Expression (23) is not satisfied and Expression (25) is also not satisfied (S502-No, S510-No), the upper left vertex (xtl, ytl) is on the left side of the boundary indicated by Expression (22) and is not equal to Expression (22). It will be located below the boundary shown by (24). In this case, the protruding place determination processing unit 134 determines that (xtl, ytl) exists in the captured image, and sets flagtl to “0” (S514).

  The protruding place determination processing unit 134 performs protruding place determination processing on other vertices of the document. The protruding place judging processing for the lower left vertex (xbl, ybl) is performed in steps S522 to S534, the protruding place judging processing for the upper right vertex (xtr, ytr) is performed in steps S542 to S554, and the protruding place determining processing for the lower right vertex (xbr, ybr) is performed. Are S562 to S564.

  As shown in the figure, although the boundary formula used for the determination is different, it is basically the same as the process for the upper left vertex. Note that the straight line on the right boundary after the perspective transformation can be expressed by Expression (26), and the straight line on the lower boundary can be expressed by Expression (27).

  As shown in FIG. 8, when the lower left of the document protrudes from the imaging range, by performing a protruding place determination process on the image after the perspective transformation of the captured image, as shown in FIG. It is determined that the vertex (xbl, ybl) at the lower left of the document protrudes to the left.

  FIG. 23 is a flowchart illustrating a detailed example of the non-inspection area setting process (S600 in FIG. 10) by the non-inspection area setting unit 138.

  First, the non-inspection area setting unit 138 determines whether or not the flag flagtl indicating the protruding place of the upper left vertex (xtl, ytl) is “0” (S602). When the flagtl is not “0” (S602-Yes), the non-inspection area setting unit 138 performs an upper left non-inspection unit setting process (S604). By this processing, (xtl1, ytl1) and (xtl2, ytl2) are extracted as the boundary coordinates of the upper left protruding portion.

  Thereafter, the non-inspection area setting unit 138 proceeds to the processing for the lower left vertex (xbl, ybl). The non-inspection area setting unit 138 determines whether the flag flagbl indicating the protruding place of the lower left vertex (xbl, ybl) is “0” (S606). If the flagbl is not “0” (S606-Yes), the non-inspection area setting unit 138 calculates the boundary coordinates of the upper left protruding part (S608). By this processing, (xbl1, ybl1), (xbl2, ybl2) are extracted as the boundary coordinates of the lower left protruding portion.

  Thereafter, the non-inspection area setting unit 138 proceeds to the process for the upper right vertex (xtr, ytr). The non-inspection area setting unit 138 determines whether the flag flagtr indicating the protruding position of the upper right vertex (xtr, ytr) is “0” (S612). If the flagtr is not “0” (S612-Yes), the non-inspection area setting unit 138 calculates the boundary coordinates of the upper right protruding part (S614). By this processing, (xtr1, ytr1) and (xtr2, ytr2) are extracted as the boundary coordinates of the upper right protruding portion.

  Thereafter, the non-inspection area setting unit 138 proceeds to the processing for the lower right vertex (xbr, ybr). The non-inspection area setting unit 138 determines whether the flag flagbr indicating the protruding place of the lower right vertex (xbr, ybr) is “0” (S616). If the flagbr is not “0” (S616-Yes), the non-inspection area setting unit 138 calculates the boundary coordinates of the lower right protruding part (S618). By this processing, (xbr1, ybr1) and (xbr2, ybr2) are extracted as the boundary coordinates of the lower right protruding portion. Thereafter, the non-inspection area setting unit 138 starts a non-inspection unit setting process.

  24 and 25 show a specific example of the lower left non-inspection unit setting process. Here, FIG. 24 is a flowchart of the lower left non-inspection section setting process (S608). FIG. 25 (A) is a diagram showing the boundary coordinate position, and FIG. 25 (B) shows the equations and intersection coordinates used in the processing step for the lower left shown in FIG. 24 in the upper left, upper right and lower right corners. It is a chart which shows the correspondence at the time of applying.

  The non-inspection area setting unit 138 specifies the calculation of the boundary coordinates of the lower left protruding part. Specifically, it is determined whether the flag flagbl indicating the protruding place of the lower left vertex (xbl, ybl) is "1" or "2" (S620). If the flagbl is "1" (S620-Yes), the intersection (xbl1, xbl1) of yl and at1.times.x + bt1.times.y + ct1 = 0 is calculated because the lower left vertex protrudes to the left of the captured image (S624). . Thereafter, the intersection (xbl2, xbl2) of yb and at1 × x + bt1 × y + ct1 = 0 is calculated (S626), and the lower left non-inspection unit setting process ends. If flagbl is “2” (S620-No), the intersection (xbl1, xbl1) of yl and y + ct4 = 0 is calculated because the lower left vertex protrudes below the captured image (S624). Thereafter, the intersection (xbl2, xbl2) of yb and y + ct4 = 0 is calculated (S626), and the lower left non-inspection unit setting process ends.

  The same applies to the process of calculating the boundary coordinates of the upper left protruding portion (S604), the process of calculating the boundary coordinates of the upper right protruding portion (S614), and the process of calculating the boundary coordinates of the lower right protruding portion (S618). However, the equations and the intersection coordinates used in the processing step for the lower left shown in FIG. 24 are changed as shown in FIG.

  When the non-inspection area setting unit 138 completes the pre-processing related to the overflow of the captured image as described above, the non-inspection area setting unit 138 stores information indicating the area of the overhang portion (non-inspection area) in the data storage unit 300 (FIG. 6). S144).

  Next, the resolution conversion unit 214 of the reference image processing unit 210 adjusts the resolution of the reference image so that the size of the rectangular reference image is equal to the size of the image of the rectangular document portion in the captured image to be inspected. Is converted (S152). Next, the blurring processing unit 216 performs a blurring process (Gaussian filter process) on the resolution-converted reference image (S154).

  Next, the non-inspection area deletion processing unit 218 reads out the non-inspection area information specified by the non-inspection area specifying unit 130 from the data storage unit 300, and performs the shading processing by the shading processing unit 216 on the reference image. The non-inspection area is set for the reference image by invalidating the data of the non-inspection area in (S156). The non-inspection area setting unit 138 stores the processed reference image in the data storage unit 300.

  Next, the alignment processing unit 232 performs alignment processing between the reference image in which the non-inspection area portion is invalidated and the captured image subjected to the preprocessing by the image reading unit 100 (S158). Next, the defect abnormality detection processing unit 234 calculates the difference between the two images (S160), and detects the presence or absence of a defect or an abnormality in the captured image based on the difference result obtained by the difference processing (S162).

  The resolution conversion unit 214 of the image inspection processing unit 200 reads out the reference image, performs resolution conversion, performs the blurring processing on the reference data, and performs shading correction processing, inverse geometric transformation (reverse perspective transformation) processing, and non-inspection unit processing. After the alignment processing (S122) between the captured images subjected to the setting processing, the comparison is performed, and the defect abnormality detection processing (S123) is performed.

  As described above, in the image inspection apparatus 5 of the first embodiment, even when a part of the document of the captured image is out of the imaging range due to the tilt scan (see FIGS. By estimating the vertices and setting the non-inspection part, the protruding part can be excluded from the inspection target and compared with the reference image. As a result, the accuracy of comparison / inspection between the captured image and the reference image is improved, regardless of the state of discharge of the document, and inspection for image defects and abnormal image quality with higher inspection accuracy can be performed.

  For example, as shown in FIG. 8, even when the lower left vertex of the document protrudes from the imaging range, the protruding vertex is estimated and the non-inspection part is set to exclude the protruding part from the inspection target. And can be compared with the reference image.

  As can be seen from FIGS. 5 and 7, even if the coordinates of the vertices are specified, error processing may be performed as a result. The reason why the vertex coordinates are obtained even in the case of error processing is that it is necessary to know the area size of the document when performing resolution conversion in the resolution conversion unit 214.

<< 2nd Embodiment >>
FIG. 26 is a block diagram showing a detailed example of the second embodiment of the image inspection apparatus 5 shown in FIG. In the second embodiment, a non-inspection area that protrudes from an imaging range of a document portion based on a trapezoidal captured image without performing perspective conversion processing on a captured image in which a part of the document protrudes from the imaging range. When non-inspection area information for identifying the image is obtained and the image inspection is performed by comparing the trapezoidal captured image with the reference image, first, the reference image is made to be the same as the trapezoidal shape of the captured image. It is characterized in that a reverse perspective transformation process is performed, and thereafter, image inspection is performed by referring to the previously obtained non-inspection area information and excluding the non-inspection area of the document from the inspection target. This will be specifically described below.

  The configuration of the first embodiment is different from that of the first embodiment in that the perspective conversion unit 116 is removed from the image reading unit 100 so that the perspective conversion process is not performed on the captured image. The configuration of the first embodiment is different from the configuration of the first embodiment in that a reverse perspective transformation unit 212 that performs a geometric transformation process (reverse perspective transformation process) on the reference image so as to have the same shape as the trapezoidal shape of the captured image is provided. .

  The inverse perspective transformation unit 212 obtains, for example, an inverse transformation expression (the expression is omitted) of the above-described expressions (3) and (4) used in the perspective transformation process of returning the trapezoidal shape of the captured image to a rectangular shape. Then, by substituting each pixel of the reference image, the rectangular reference image matches the trapezoid of the captured image.

  FIG. 27 is a flowchart illustrating an outline of a processing procedure in the image inspection apparatus 5 according to the second embodiment. The basic processing procedure is the same as that of the image inspection apparatus 5 of the first embodiment, except that the perspective transformation processing is not performed on the captured image, In that a reverse perspective transformation process is performed. Similar processing procedures are denoted by the same step numbers.

  The shading correction unit 112 performs shading correction on the image captured by the camera head 102 (S134), further performs optical distortion correction, and stores the processed image in the data storage unit 300 (S136).

  Thereafter, the non-inspection area specification processing unit 120 immediately executes the non-inspection area specification processing based on the trapezoidal captured image according to a procedure described later, and outputs information indicating the protruding area (non-inspection area) to the data. It is stored in the storage unit 300 (S144).

  Next, the reverse perspective transformation unit 212 of the reference image processing unit 210 performs a reverse perspective transformation process on the rectangular reference image so as to match the trapezoidal shape of the captured image (S150). The resolution conversion unit 214 converts the resolution of the reference image (S152) such that the size of the reference image (trapezoid) is equal to the size of the image (trapezoid) of the original portion of the captured image to be inspected (S152). Next, the blurring processing unit 216 performs blurring processing on the resolution-converted reference image (S154).

  Hereinafter, similarly to the first embodiment, the non-inspection area deletion processing unit 218 reads out the non-inspection area information specified by the non-inspection area identification unit 130 from the data storage unit 300, and the unsharp processing unit 216 performs the shading. The non-inspection area is set for the reference image by invalidating the data of the non-inspection area part in the processed reference image (S156). The alignment processing unit 232 performs alignment processing between the reference image in which the non-inspection area portion is invalidated and the captured image subjected to the preprocessing by the image reading unit 100 (S158). The defect / abnormality detection processing unit 234 calculates the difference between the two images (S160), and detects the presence or absence of a defect or abnormality in the captured image based on the difference result obtained by the difference processing (S162).

  FIG. 28 is a diagram illustrating an example of an image handled in the second embodiment. Here, FIG. 28A is a diagram illustrating a result of an image captured by the camera head 102 in a state in which a part of the document protrudes from the imaging range and is placed, and is illustrated in FIG. Is the same as The document portion in the image data before the perspective transformation processing captured by the camera head 102 has a trapezoidal shape as shown in FIG.

  FIGS. 28B and 28C are diagrams illustrating the processing of the edge extraction unit 122. FIG. The edge extraction unit 122 performs an edge extraction process on the trapezoidal captured image optically corrected by the optical distortion correction unit 114. As in the first embodiment, the edge extraction process is performed by performing a search using the prewitt operator from the upper, left, lower, and right directions. By taking the sum of each pixel of the edge extraction image obtained by searching from the upper, left, lower, and right directions, a combined edge extraction image shown in FIG. 28B is obtained.

  The effective edge portion detection unit 124 performs the effective edge portion detection processing based on the edge extracted image shown in FIG. 28B, thereby obtaining the edge extracted image shown in FIG. 28C. In FIG. 28C, a portion indicated by four straight lines (thick lines in the drawing) is an edge effective portion.

  Next, a processing procedure of a non-inspection area specifying process of the non-inspection area specifying processing unit 120 according to the second embodiment will be described. Since the basic processing procedure is the same as that of the first embodiment, only the part relating to the second embodiment will be described with reference to the step numbers of the flowchart of the first embodiment.

  First, with reference to FIGS. 20 and 21, a description will be given of the edge-effective-portion straight-line calculation process by the edge-effective-portion straight-line calculation unit 126 and the document vertex setting process by the document vertex setting processing unit 132.

  First, similarly to the first embodiment, the edge-effective-portion straight-line calculation unit 126 calculates the vertex of the document based on the edge-effective portion detected by the edge-effective-portion detection unit 124. A calculation process is performed (S402). That is, a straight line yt of the upper edge effective portion, a straight line formula yb of the lower edge effective portion, a straight line formula yl of the left edge effective portion, and a straight line expression yr of the right edge effective portion are calculated.

  The document vertex setting processing unit 132 uses the straight line formula calculated by the edge effective part straight line formula calculation unit 126 to set the vertex (xtl, ytl) at the upper left of the document, the vertex (xbl, ybl) at the lower left of the document, and the upper left corner of the document. The vertices (xtr, ytr) and the upper left vertex (xbr, ybr) of the document are calculated (S420 to S450).

  FIG. 29 is a flowchart illustrating a detailed example of the protruding place determination processing (corresponding to FIG. 22 of the first embodiment) by the protruding place determination processing unit 134 according to the second embodiment. FIG. 30 is a diagram illustrating an example of the protruding place determination processing performed by the protruding place determination processing unit 134.

  Since the processing of the second embodiment is basically the same as that of the first embodiment, the step numbers of the processing to be described later are marked with a "@" mark in the first embodiment for simplicity of comparison. The same numbers as in the embodiments are used.

  The protruding place determination processing unit 134 first determines whether the upper left vertex (xtl, ytl) of the document protrudes to the left side (S # 502). From FIG. 30A, the straight line on the left boundary can be expressed by Expression (28). Therefore, whether the upper left vertex (xtl, ytl) is located on the left side of the straight line of the left boundary is determined by substituting (xtl, ytl) into the equation (28) as shown in the equation (29). It can be determined whether it is larger or smaller. If (xtl, ytl) is located on the left side of the boundary shown by Expression (28), the condition of Expression (29) is satisfied.

  Similarly, the protruding place determination processing unit 134 determines whether the top left vertex (xtl, ytl) of the document protrudes upward (S # 504, S # 510). As shown in FIG. 30A, the straight line on the upper boundary can be expressed by equation (30). Therefore, whether the upper left vertex (xtl, ytl) is positioned above the straight line of the upper boundary is determined by substituting (xtl, ytl) into equation (30) as shown in equation (31). It can be determined whether it is larger or smaller. If (xtl, ytl) is located above the boundary shown in Expression (30), the condition of Expression (31) is satisfied.

  When Expression (29) is satisfied and Expression (31) is also satisfied (S （502-Yes, S ＄ 504-Yes), the protruding place determination processing unit 134 determines that the upper left vertex (xtl, ytl) is the expression ( 28) is located on the left side of the boundary line shown by the above expression and above the boundary line shown by the expression (30). In this case, the length of any one of the sides of the document is not known, that is, there is no one that fits the entire side within the imaging range, so the protruding place determination processing unit 134 performs error processing (S $ 506).

  When equation (29) is satisfied and equation (31) is not satisfied (S ＄ 502-Yes, S ＄ 504-No), the upper left vertex (xtl, ytl) is located on the left side of the boundary shown in equation (28). And it will be located below the boundary line shown by Formula (30). In this case, the protruding place determination processing unit 134 determines that the document protrudes to the left side, and sets flagtl to “1” (S # 508).

  When Expression (29) is not satisfied and Expression (31) is satisfied (S ＄ 502-No, S ＄ 510-Yes), the upper left vertex (xtl, ytl) is located on the right side of the boundary indicated by Expression (28). And it will be located above the boundary shown by the equation (30). In this case, the protruding place determination processing unit 134 determines that the document protrudes upward, and sets flagul to “2” (S # 512). Further, when Expression (29) is not satisfied and Expression (31) is also not satisfied (S ＄ 502-No, S ＄ 510-No), the left side of the boundary indicated by Expression (28) and the expression (30) are used. It will be located below the indicated boundary line. In this case, the protruding place determination processing unit 134 determines that (xtl, ytl) exists in the captured image, and sets flagtl to “0” (S # 514).

  The protruding place determination processing unit 134 performs protruding place determination processing on other vertices of the document. The protruding place judging processing for the lower left vertex (xbl, ybl) is S ＄ 522 to S ＄ 534, the protruding place judging processing for the upper right vertex (xtr, ytr) is S ＄ 542 to S ＄ 554, and the lower right vertex (xbr , Ybr), the out-of-place determination processing is S $ 562 to S $ 564.

  As shown in the figure, although the boundary formula used for the determination is different, it is basically the same as the process for the upper left vertex. Note that the straight line on the right boundary can be expressed by Expression (32), and the straight line on the lower boundary can be expressed by Expression (33).

  As shown in FIG. 30A, when the lower left of the document is out of the imaging range, by performing the protruding place determination process on the captured image, the lower left vertex (xbl, ybl) of the document is protruded to the left. It is determined that there is.

  FIG. 31 is a flowchart illustrating a specific example of the boundary coordinate calculation processing of the protruding part (corresponding to FIG. 24 of the first embodiment) by the boundary part specifying processing unit 136 of the second embodiment. Here, FIG. 31 shows a flowchart of the boundary coordinate calculation processing of the lower left protruding portion. FIG. 32 is a chart showing a correspondence relationship in a case where the equations and intersection coordinates used in the processing step for the lower left shown in FIG. 31 are applied to the upper left, upper right, and lower right.

  A detailed example of the process of calculating the boundary coordinates of the protruding portion (S600 in FIG. 10) by the non-inspection area setting unit 138 is the same as that of the first embodiment shown in FIG.

  When performing the boundary coordinate calculation processing of the lower left protruding portion (S ＄ 608), when the flagbl is “1”, the protruding location determination processing section 134 determines whether the protruding location determination processing section 134 has a straight line yl and a left boundary line x + cq1 = 0. The intersection (xbl1, ybl1) is calculated (S ＄ 622). Thereafter, the protruding place determination processing unit 134 calculates the intersection (xbl2, ybl2) of the straight line yb and the left boundary line y + cq1 = 0 (S ＄ 624), (xbl, ybl), (xbl1, ybl1), ( xbl2 and ybl2) are set as non-inspection units (S # 626). If flagbl is “2”, the protruding place determination processing unit 134 calculates an intersection (xbl1, ybl1) of the straight line yl and the lower boundary line x + cq4 = 0 (S ＄ 632). Thereafter, the intersection (xbl2, ybl2) of the straight line yb and the lower boundary line x + cq4 = 0 is calculated (S ＄ 634), and the area between (xbl, ybl), (xbl1, ybl1), and (xbl2, ybl2) is not inspected. It is set as a copy (S # 636).

  The same applies to the process of calculating the boundary coordinates of the upper left protruding portion (S604), the process of calculating the boundary coordinates of the upper right protruding portion (S614), and the process of calculating the boundary coordinates of the lower right protruding portion (S618). However, the equations and intersection coordinates used in the processing step for the lower left shown in FIG. 31 are changed as shown in FIG.

  As described above, in the image inspection device 5 of the second embodiment, the captured image is converted into an image viewed from the front (that is, returned to a rectangular image) and compared with the reference image which is the original image. Rather, the reference image is subjected to reverse perspective transformation processing so as to match the shape of the captured image (generally trapezoidal shape) resulting from the imaging state, and none of the rectangular images is trapezoidal (in the previous example, trapezoidal). Are compared. The reverse perspective transformation processing for the reference image does not require interpolation processing of finding pixel values of more pixel points from fewer pixel points, so that the presence or absence of an image defect can be inspected more accurately.

  Also, at this time, when a part of the original of the captured image protrudes due to the tilt scan, the vertex coordinates of the protruding part are estimated and the non-inspection part is set without performing the geometric transformation process on the captured image. This enables comparison with the reference image.

<<< 3rd Embodiment >>>
FIG. 33 is a view for explaining the outline of the document range specifying process in the third embodiment of the image inspection apparatus 5 shown in FIG. As the configuration of the image inspection apparatus 5 according to the third embodiment, the same configuration as that of the first embodiment is used. As in the first embodiment, the image inspection apparatus 5 according to the third embodiment performs the perspective transformation process on the photographed image in which the vertices of the substantially rectangular original are out of the imaging range, and performs the perspective transformation process. Based on the rectangular captured image, non-inspection area information for identifying a non-inspection area protruding from the imaging range of the original portion is obtained, and the perspective-transformed captured image is compared with the reference image to perform image inspection. When performing the image inspection, the image inspection is performed by excluding the non-inspection area of the document from the inspection target with reference to the previously obtained non-inspection area information.

  The third embodiment is different from the first and second embodiments in the method of obtaining a linear expression indicating the edge of a document and the method of setting a portion of the document outside the imaging range as a non-inspection unit. Specifically, an equation of a straight line forming the edge of the rectangular document is obtained by using a method such as edge tracking or statistical processing, and the number of obtained straight lines is any of four, three, two or less. It is characterized in that the non-inspection area setting processing is switched depending on whether the process is performed. Thereby, it is possible to cope with a captured image in a protruding state which cannot be dealt with by the processing procedure of the first embodiment.

  For example, the image inspection apparatus 5 according to the third embodiment uses a well-known technique of edge tracking as a method of calculating a linear expression of a document edge, and a predetermined number of edge pixels are substantially continuously formed in a substantially linear shape. When it is detected, it is assumed that the portion is a straight line portion forming the edge of the document. Alternatively, a straight line forming the edge of the document may be obtained by using a statistical method. As shown in FIG. 33A, it is possible to detect a straight line that forms the periphery of the document even when the document is inclined.

  Thereafter, as shown in FIG. 33B, the image inspection apparatus 5 first switches the process depending on whether or not four linear formulas forming the document margin have been detected (S1). If four straight-line equations can be detected (S1-Yes), all vertex coordinates can be specified even if some vertices are outside the imaging range. On the other hand, if four linear expressions cannot be detected by edge tracking or the like (S1-No), the pixel signal generation unit 5 switches the processing depending on whether three linear expressions forming the document margin have been detected (S1). S2). When three linear expressions can be detected, there is a case where the entirety of one side is within the imaging range or only one vertex is within the imaging range, but in any case, two vertices (shown in the drawing) Since the mark (●) can be set, the pixel signal generation unit 5 first sets them (S1).

  Next, the pixel signal generation unit 5 calculates the distance between the two coordinates of the vertices that have already been set (S4), and calculates the distance and the ratio (size ratio) between the respective edges of the substantially rectangular shape of the reference image. The coordinates of two unknown vertices (marked with a circle in the figure) facing the two already set vertices are estimated (specified) (S5). Then, the remaining straight line formulas that define the sides outside the imaging range are specified from the obtained two vertex coordinates (S6).

  When the four linear equations can be specified in this way (S1-Yes or S6), the pixel signal generator 5 next calculates the intersection coordinates from the four linear equations (S7). Then, from the four intersection coordinates, four straight lines are associated with each side of the document, and which vertex of the document the four intersection coordinates are specified (S8). Thus, the position of the vertex of the document that protrudes from the imaging range can be specified.

  On the other hand, when only two or less straight lines can be detected by edge tracking or the like (S2-No), only one vertex or less exists in the imaging range. In this case, it is very difficult, if not impossible, to specify the four vertices, and the document area cannot be determined. Therefore, the pixel signal generation unit 5 stops the process as an error (S9).

  FIG. 34 is a diagram illustrating an outline of a non-inspection area specifying process according to the third embodiment. Here, a state where the image is protruded to the left end side of the imaging range will be described as an example.

  In the state in which a document that can be handled is out of the imaging range in the processing of the third embodiment, three or four straight lines forming the edge of the document are obtained. When this is viewed from the upper, lower, left and right edges of the imaging range, as shown in FIG. 34A, when one vertex protrudes from one edge, as shown in FIG. 34B. When one vertex protrudes from two edges, that is, when it protrudes toward the corner of the imaging range, and as shown in FIG. 34C, one vertex protrudes from one or two edges. When two adjacent vertices protrude, they can be roughly classified into three modes.

  Here, FIGS. 34 (A) and 34 (B) each show a case where one vertex protrudes, and can be regarded as a state of the same concept in the non-inspection area setting process. In such a case, the coordinates of two intersections between two straight lines extending from the vertex of the original protruding from the imaging range and the edge of the imaging range to be protruded are obtained, and the coordinates of the two intersections and the original document are determined. The triangular area S1 formed by the vertices may be set as the non-inspection area.

  However, in the state shown in FIG. 34B, since there are two edges of the target imaging range, two intersection coordinates are obtained for each edge, and as a result, two triangular regions are obtained. S11 (reference element "l" means left) and S1b (reference element "b" means bottom) are set as non-inspection areas. Although an overlapping portion occurs between S1a and S1b, there is no inconvenience. Note that S11 can be regarded as S1 viewed from the left edge, and S1b can be regarded as S1 viewed from the lower edge.

  In other words, the process of setting one triangular area S1 protruding from a certain edge of the imaging range as a non-inspection area is performed for the four upper, lower, left, and right edges, thereby protruding from the imaging range. All triangular areas can be reliably specified as non-inspection areas.

  On the other hand, in the state shown in FIG. 34C, with respect to the two document vertices on the common document edge protruding from the edge of the target imaging range, the remaining one straight line extending from each of the two vertices is The coordinates of the intersection between the edge of the imaging range to be protruded and the edges of the two originals protruding from the imaging range may be set as the non-inspection area. .

  In the above-described processing, when one vertex protrudes from the edge of the imaging range of interest, a triangular area is protruded, and when two vertices protrude, a quadrangular area is protruded. Although the processing contents are switched in accordance with the protruding state, such as setting each in a non-inspection area, these can be handled collectively.

  For example, the coordinates of the intersection between each of the remaining one straight line extending from each of the two document vertices on the common document edge and the edge of the imaging range to be protruded are determined. The area surrounded by the two document vertices protruding from the range may be set as the non-inspection area. In this case, the area S3 in FIG. 34 (A) and the areas S31 and S3b in FIG. 34 (B) are also set as non-inspection areas. It becomes a practically meaningless area for the purpose.

  FIG. 35 is a diagram for explaining an outline of a processing procedure for realizing the above processing contents. Here, FIG. 35A is a flowchart illustrating a procedure of the boundary coordinate specifying process by the boundary portion specifying processing unit 136. In FIG. 35A, the edge of the document is typically described by reference numeral "A", and the linear expression y is indicated by "ya" using the lowercase letter "a" of reference numeral "A". FIGS. 35 (B) and 35 (C) show a straight line formula for defining a protruding area for each of the upper, lower, left and right edges used in this processing, and an intersection between an original vertex and a predetermined straight line and an edge of the imaging range. It is a figure showing each correspondence of coordinates. 34A to 34C also show the coordinates shown in FIGS. 35B and 35C.

  The boundary part specification processing unit 136 of the image inspection apparatus 5 first specifies the coordinates of the intersection points αa and βa between the two boundary expressions yα and yβ related to the linear expression ya on the side A of the document and the edge of the imaging range. (S11). Here, the two boundary expressions yα and yβ are equivalent to the remaining linear expressions orthogonal to the linear expression ya extending from the two document vertices on the linear expression ya. For example, in the case of the left side L (Left) of the document, a straight line formula yt of the upper side T (Top) and a straight line formula yb of the lower side B (bottom) are used.

  Next, the boundary part specification processing unit 136 determines whether or not the obtained intersection coordinates are outside the range of the document (S12). When the intersection coordinates are outside the range of the document, the document vertices Aα and Aβ (either but not both) on the boundary formula (that is, the linear formula) including the intersections αa and βa are within the imaging range. Become. The intersection in such a state is hereinafter referred to as an external dividing point. On the other hand, when the coordinates of the intersection are within the range of the document, the document vertices Aα and Aβ on the boundary formula (that is, the linear formula) including the intersections αa and βa (both may be both) are outside the imaging range. It will be in. The intersection in such a state is hereinafter referred to as an internally dividing point.

  When the intersection coordinates αa and βa are outside divisions outside the document range (S12-Yes), the boundary portion specification processing unit 136 combines the intersection coordinates of the outside division and the corresponding vertex coordinates as the intersection coordinates Aa. (S13). Here, the corresponding vertex coordinates are either one of the document vertices Aα and Aβ on the boundary formula (that is, the linear formula) including the intersection coordinates of the external dividing points. The intersection coordinates Aa mean the coordinates of the intersection between the straight line ya including the vertex of the document and the corresponding edge of the imaging range.

  Thereafter, the boundary part specifying processing unit 136 sets a triangle defined by the remaining intersection coordinates, the vertex coordinates (the other of Aα and Aβ), and the intersection coordinates Aa in the non-inspection unit (S14). By the processes of steps S12-Yes to S14, S1, S1a, and S1b shown in FIGS. 34A and 34B can be specified.

  On the other hand, when the intersection coordinates αa and βa are internal dividing points within the document range (S12-No), the boundary part specifying processing unit 136 determines that the two document vertices Aα and Aβ on the straight line ya and the two intersections A quadrangle defined by the coordinates αa and βa is set in the non-inspection section (S15). By the processing of steps S12-No and S15, S2 shown in FIG. 34C can be specified.

  Note that by performing the processing of step S15 for all, the area surrounded by the two intersection coordinates αa and βa and the two document vertices Aα and Aβ protruding from the imaging range can be set as the non-inspection area. . This is the same as that described in the description of FIG. Therefore, it is not necessary to perform the processing of steps S12, S13, and S14.

  FIG. 36 and FIG. 37 are examples of a captured image showing a protruding state of a document that can be handled by the processing of the third embodiment. FIG. 38 is an example of a captured image showing a protruding state of a document that cannot be handled by the processing of the third embodiment. The protruding state of the manuscript that can be handled by the processing of the third embodiment is a case where the number of detected straight lines is four and a case where the number is three. If the number of detected straight lines is two or less, the processing is stopped as an error.

  For example, FIG. 36 shows an example of the relationship between a document and an imaging range when four straight lines are detected (when four sides are detected). Two adjacent vertices of the document form a corner with respect to the imaging range. It is in a state where it extends beyond the imaging range to be formed in two directions. Such a state includes a state in which the upper and lower edges of the captured image area after perspective transformation protrude from the upper end and the right end (FIG. 36A), and a state in which the upper and lower ends protrude at the upper left corner (FIG. 36A). (FIG. 36 (B)), a state where the lower end and the right end protrude at the lower right corner (FIG. 36 (C)), and a state where the lower end and the left end protrude at the lower left corner (FIG. 36 (D)). ), Are typical.

  FIG. 37 shows an example of the relationship between the document and the imaging range when three straight lines are detected (when three sides are detected). At least two adjacent vertices of the document protrude from the edge of the imaging range. It is in the state that it was. In such a state, first, two adjacent vertices of a document protrude from one edge of the imaging range. In this state, the vertices of two adjacent originals are both located outside the straight line that forms one edge of the imaging range.

  For example, the upper left and lower left vertices of the document protrude to the left side of the captured image range after the perspective transformation (FIG. 37A), and the upper left and upper right vertices of the document protrude upward (FIG. 37 ( B)), the upper right and lower right vertices of the document protrude to the right (FIG. 37 (C)), and the lower left and lower right vertices of the document protrude to the lower side (FIG. 37 (D) )) Are typical.

  On the other hand, as another state where three straight lines are detected (when three sides are detected), there is a state in which two adjacent document vertices protrude from two edges of the imaging range. In this state, two adjacent document vertices are located on the corner side of the imaging range, one of them is located outside one straight line that forms the edge of the corner, and the other vertex is located on one side. Is located outside of the other straight line that forms an edge and intersects with the straight line.

  For example, a state in which the two vertices on the left side of the document protrude substantially toward the upper left corner of the captured image area after the perspective transformation (FIG. 37E), and a state in which the two vertices on the right side of the document protrude substantially toward the upper right corner side (FIG. 37 (F)), a state in which the two vertices on the right side of the document protrude substantially toward the lower right corner (FIG. 37 (G)), and a state in which the two vertices on the left side of the document protrude substantially toward the lower left corner. The state (FIG. 37H) is representative.

  On the other hand, (A) to (D) of FIG. 38 show an example in which the number of detected straight lines is two or less, and only one vertex exists in the imaging range. In such a case, it is not possible to specify all of the vertex coordinates of the rectangular document based on the obtained straight-line intersection, so the image inspection apparatus 5 stops the processing as an error. Note that, even in such a case, although there is a problem in accuracy, it is possible to specify all the vertex coordinates from, for example, a magnification relationship. However, since this requires complicated processing, it is not adopted in this embodiment, and an error is generated as described above.

  Hereinafter, an apparatus configuration and a processing procedure for realizing the processing of the third embodiment will be described in detail with reference to specific examples.

  FIG. 39 is a block diagram illustrating a configuration example of the non-inspection area specifying processing unit 120 according to the third embodiment. The non-inspection area identification processing unit 120 according to the third embodiment is different from the first embodiment in that the configuration of the edge valid part detection unit 124 is different from that of the first embodiment in that it performs edge tracing on the edge portion extracted by the edge extraction unit 122 And an edge pixel counting unit 124b that counts the number of edge pixels continuous with the reference edge pixel. When the edge pixel counting section 124b counts the edge pixels almost continuously in a straight line at a predetermined number or more, the edge effective portion detection section 124 detects that portion as an edge effective section.

  Further, the non-inspection area specifying processing unit 120 of the third embodiment is different from the first embodiment in the configuration of the non-inspection area specifying unit 130, and includes a protruding vertex coordinate estimation processing unit 135. The protruding vertex coordinate estimation processing unit 135 calculates the coordinates of the remaining vertices when the number of straight lines forming the document margin existing in the imaging range is three and all the vertices of the document outside the imaging range cannot be calculated. Infer.

  The non-inspection area specifying unit 130 refers to the document vertices within the imaging range obtained by the document vertex setting processing unit 132 and the document vertices estimated by the protruding vertex coordinate estimation processing unit 135, and determines the vertex coordinates in the imaging range. A boundary portion between the document vertex determined to be out of the image and the imaging range is specified by the boundary portion specification processing unit 136, and the document coordinates are determined by referring to the boundary coordinates and the document vertex outside the imaging range. The outside of the range is set as the non-inspection area.

  FIG. 40 is a flowchart illustrating an outline of a processing procedure in the non-inspection area specifying processing unit 120 according to the third embodiment. The image inspection device 5 basically executes substantially the same procedure as the processing procedure shown in FIG. 10 of the first embodiment.

  For example, first, similarly to step S200 of the first embodiment, the edge extraction unit 122 extracts an edge component from the captured image perspective-transformed by the perspective transformation unit 116 (S4000). As a result, an edge-extracted image as shown in FIGS. 9A to 9D is obtained.

  Next, the edge valid part detection unit 124 searches for edge pixels from each of the upper, lower, left, and right edges of the imaging range with reference to the edge information extracted for each of the up, down, left, and right directions in step S4000, thereby obtaining one edge. A pixel is specified (this process is referred to as an edge effective portion detection process) (S4002). In addition, the edge-effective-portion straight-line calculation unit 126 determines a vertex coordinate of the document based on the edge-effective portion detected by the edge-effective-portion detection unit 124. (This process is referred to as an edge effective part straight line formula calculation process) (S4004).

  Although details will be described later, the edge effective portion specifying process and the straight line expression calculating process are different from those of the first embodiment. Hereinafter, the edge effective part detection processing by the edge effective part detection unit 124 and the edge effective part linear expression calculation processing by the edge effective part linear expression calculation unit 126 are collectively referred to as a document range identification process.

  Next, the document vertex setting processing unit 132 specifies the vertex coordinates of the document in the captured image with reference to the linear equation calculated by the edge effective unit linear equation calculating unit 126 (S4006).

  The protruding place determination processing unit 134 determines whether or not the vertex coordinates set by the document vertex setting processing unit 132 protrude from the imaging range, and how many straight lines forming the document edge existing in the imaging range. A determination is made (S4008).

  The protruding vertex coordinate estimation processing unit 135 estimates the remaining vertices when the number of straight lines forming the document margin existing in the imaging range is three and all protruding vertices cannot be calculated as it is (S4010). .

  Next, similarly to the first embodiment, the boundary part specifying processing unit 136 determines a boundary part between the document vertex and the image vertex whose vertex coordinates are determined to protrude from the image capturing range by the protruding place determination processing unit 134. It is specified (S4012). At this time, the boundary portion specification processing unit 136 determines which edge of the captured image is in contact with the coordinates of both ends of the valid edge portion of the document detected by the edge valid portion detection unit 124, and determines the determination result. Then, the coordinates of the intersection between the document margin and the imaging range are specified by determining the direction in which the document protrudes.

  The non-inspection area setting unit 138 outputs the document based on the coordinate information indicating the specified boundary portion and the coordinate information of the document vertex whose vertex coordinates are determined to be outside the imaging range by the protruding place determination processing unit 134. The non-inspection area protruding from the imaging range is defined (S4014). The non-inspection area setting unit 138 registers information indicating the defined non-inspection area in the data storage unit 300. Hereinafter, the boundary part specifying processing by the boundary part specifying processing unit 136 and the non-inspection part setting processing by the non-inspection area setting unit 138 are collectively referred to as non-inspection area specification processing.

  FIG. 41 is a diagram illustrating a relationship between a search start point and a search direction when the edge valid part detection unit 124 in the third embodiment searches for an edge pixel, and the first edge pixel detected by the edge valid part detection unit 124 as a start point. FIG. 7 is a diagram illustrating a relationship between directions in which an edge valid part linear expression calculation unit 126 performs edge tracking.

  In the third embodiment, as shown in FIG. 41A, the edge effective portion detection unit 124 sets the horizontal pixel position as i and the vertical pixel position as j, and sets the left end of the imaging range after the perspective transformation as An edge pixel is searched from each of the upper end, the right end, and the lower end. Here, FIG. 41A shows a case where the original is inclined with respect to the imaging range, but is completely within the imaging range. In the third embodiment, it is assumed that processing is performed using a captured image range (the maximum horizontal size ph and the maximum vertical size pv) after perspective transformation as a scan range.

  In this case, when performing a search from the left end, the edge valid part detection unit 124 sets the starting point of the search as the origin (0, 0) and first sets the horizontal direction of the captured image range after perspective transformation in the + i direction until an edge pixel is found. The process of searching up to the maximum size ph in the direction is repeated up to the maximum size pv in the vertical direction. In this way, an edge pixel at the vertex of the document indicated by a point a in the figure is found. In response to this, the edge-effective-portion straight-line-type calculating unit 126 searches for a straight line connected to the point a by performing edge tracing in a range of the upper left 90 degrees and a range of the upper right 90 degrees with reference to the point a. By searching from the lower left to the upper right to find the first edge pixel a (the document vertex if it is within the imaging range), the edge pixel a is used as the starting point, and the search is further performed upward (in a range of 180 degrees). This is to pursue two straight edge pixels forming the vertex of the document.

  Similarly, a search is performed from the upper left to the lower right to find the first edge pixel b (or the top of the document if it is within the imaging range), and further to the right (a range of 180 degrees) starting from the edge pixel b. Thus, the edge pixels that form two straight lines that form the vertex of the document are tracked. For this reason, the edge valid part detection unit 124 starts the search from the upper end with the start point of the search as the origin (0, pv), and performs a process of searching until j = 0 in the −j direction until an edge pixel is found. Repeat up to the maximum size ph in the direction. By doing so, an edge pixel at the vertex of the original indicated by a point b in the figure is found. In response to this, the edge-effective-portion straight-line-type calculating unit 126 searches for a straight line connected to the point b by performing edge tracing in a range of 90 degrees in the upper left and a range of 90 degrees in the upper right based on the point b.

  In addition, a search is made from the upper right to the lower left to find the first edge pixel c (the document vertex is located within the imaging range), and the edge pixel c is used as a starting point to search further downward (in a range of 180 degrees). , The edge pixels forming two straight lines forming the vertex of the document are tracked. For this reason, the edge valid part detecting unit 124 starts the search from the right end with the starting point of the search as the origin (ph, pv), and searches for i = 0 in the −i direction until an edge pixel is found, j Repeat until = 0. By doing so, the edge pixel at the vertex of the original indicated by the point c in the figure is found. In response to this, the edge-effective-portion straight-line calculation unit 126 searches for a straight line connected to the point c by performing edge tracing in the range of the upper left 90 degrees and the upper right 90 degrees based on the point c.

  Further, a search is performed from the lower right to the upper left to find the first edge pixel d (the document vertex is located within the imaging range), and further searching leftward (in a range of 180 degrees) with the edge pixel d as a starting point. , The edge pixels forming two straight lines forming the vertex of the document are tracked. For this reason, the edge valid part detecting unit 124 starts the search from the right end with the starting point of the search as the origin (ph, 0), and first, until the edge pixel is found, the maximum in the vertical direction of the captured image range after the perspective transformation in the + j direction. The process of searching up to the size pv is repeated until i = 0. By doing so, an edge pixel of the document vertex indicated by a point d in the figure is found. In response to this, the edge-effective-portion straight-line-type calculating unit 126 searches for a straight line connected to the point d by performing edge tracing in the range of the upper left 90 degrees and the lower left 90 degrees based on the point d.

  Charts summarizing the above processing procedures are (B) and (C) in FIG. In each processing step of the edge effective portion detection processing described later, when finding a straight line starting from the edge search direction and the first edge found, the conditions shown in FIGS. 41B and 41C are applied. And perform the processing. As shown in FIG. 41C, the edge tracking by the edge valid part linear expression calculating unit 126 starting from any one of the edge pixels a, b, c, and d found by the edge valid part detecting unit 124 is performed. In this case, the search is performed by dividing the range of 180 degrees into the range of 45 degrees.

  FIG. 42 is a diagram illustrating each direction of edge tracing by the edge tracing unit 124a of the edge valid part detecting unit 124 and an edge tracing filter. In the third embodiment, as shown in FIG. 42A, the horizontal pixel position is i and the vertical pixel position is j, each of upper right, upper left, lower left, and lower right (each 90 degree range). Is divided into two at 45 degrees, and edge tracking in eight directions as a whole is considered. However, in the edge tracing process performed by the edge valid part linear expression calculating unit 126, as shown in FIG. 41, a search for 180 degrees may be performed, and the edge tracing may be sequentially performed in four directions among eight directions. Chase.

  The edge tracking filter corresponding to each of the directions of upper right 1, upper right 2, upper left 1, upper left 2, lower left 1, lower left 2, lower right 1, lower right 2 applies a 3 × 3 pixel edge tracking filter. Therefore, for example, in the edge straight line detection processing for edge tracing in the upper right direction, the edge tracing filter shown in FIG. 42B is applied, and (i, j + 1) and (i + 1) are applied to the target pixel (i, j). , J + 1) has an edge. In the upper right two directions, the edge tracking filter shown in FIG. 42C is applied to determine whether or not there is an edge at (i + 1, j) and (i + 1, j + 1) with respect to the target pixel (i, j). Is determined.

  Similarly, in the upper left direction, the edge tracking filter shown in FIG. 42 (D) is applied, and an edge is formed at (i, j + 1) and (i−1, j + 1) with respect to the target pixel (i, j). Determine if there is. In the upper left two directions, the edge tracking filter shown in FIG. 42 (E) is applied, and the edge (i−1, j) and the edge (i−1, j + 1) for the target pixel (i, j) are obtained. Determine if there is.

  In the lower left direction, an edge tracking filter shown in FIG. 42 (F) is applied, and an edge is added to (i−1, j) and (i−1, j−1) for the target pixel (i, j). Determine if there is. In the lower left two directions, the edge tracking filter shown in FIG. 42 (G) is applied, and (i, j−1) and (i−1, j−1) are applied to the target pixel (i, j). It is determined whether or not there is an edge.

  In the lower right direction, the edge tracking filter shown in FIG. 42 (H) is applied, and the edge is located at (i, j−1) and (i + 1, j−1) with respect to the target pixel (i, j). Determine if there is. In the two lower right directions, an edge tracking filter shown in FIG. 42 (I) is applied, and edges (i + 1, j) and (i + 1, j-1) for the pixel of interest (i, j). Determine if there is.

  As described above, the edge-effective-portion straight-line calculation unit 126 divides 360 degrees into eight directions, tracks the edge pixels that form a straight line that exists within the range of 45 degrees, and, based on the result, determines the straight line. Find the formula.

  FIG. 43 is a flowchart illustrating an outline of a procedure of an edge effective portion detection process and an edge effective portion linear expression calculation process in the third embodiment. In the third embodiment, first, the edge tracking unit 124a of the edge valid part detecting unit 124 performs an edge valid part detecting process from the left end (S4101), and further, based on the result, moves the edge in the upper right direction and the upper left direction. When the tracking is performed and the edge pixels are counted in a straight line substantially continuously by a predetermined number or more by the edge pixel counting unit 124b, the portion is detected as an effective edge portion. The edge-effective-portion straight-line formula calculation unit 126 calculates a straight-line formula for this edge-effective portion (S4102). These processes are collectively referred to as a document range specifying process from the left end.

  Similarly, the edge valid part detection unit 124 performs edge valid part detection processing from the upper end (S4103), and based on the result, further performs edge tracking in the upper right direction and the lower right direction to detect an edge valid part. The edge effective part straight line formula calculation unit 126 calculates the straight line equation of the edge effective part (S4104). These processes are collectively referred to as a document range specifying process from the upper end.

  Next, the edge valid part detecting unit 124 performs an edge valid part detecting process from the right end (S4105), and based on the result, further performs edge tracking in the lower right direction and the lower left direction to detect an edge valid part. The edge effective part straight line formula calculation unit 126 calculates the straight line formula of the edge effective part (S4106). These processes are collectively referred to as a document range specification process from the right end.

  Finally, the edge valid part detecting unit 124 performs an edge valid part detecting process from the lower end (S4107), and based on the result, performs edge tracking in the upper left direction and the lower left direction to detect an edge valid part, and The valid part straight line formula calculation unit 126 calculates the straight line formula of the edge valid part (S4108). These processes are collectively referred to as a document range specifying process from the lower end.

  FIG. 44 is a flowchart illustrating a detailed example of the procedure of the document range specifying process (the edge effective portion detection process + the edge straight line calculation process). First, here, an example of the document range specifying process from the left end (S4101, S4102) will be described. As described above, the processing is performed with the scan range from (0, 0) to (ph, pv) so as to cover the captured image range after the perspective transformation.

  The edge tracking unit 124a first sets the pixel position (i, j) to the scan initial value (0, 0) (S4201), and thereafter, sets j in the right direction until scanning is performed for the vertical pixels pv of the captured image. By scanning the pixels as described later, the first edge pixel when searching from the left end is found (S4201). If j has been scanned by the number of vertical pixels of the captured image, the process of detecting an edge effective portion from the left end is terminated (S4202-Yes).

  If j has not been scanned for vertical pixels of the captured image (S4202-No), the edge tracking unit 124a determines whether the pixel (i, j) being scanned is an edge pixel (S4203). If the pixel (i, j) is not an edge pixel (S4203-No), "1" is added to the pixel position i in the horizontal direction (S4204).

  If the horizontal pixel position i at this time has not been scanned by the horizontal pixels of the captured image (S4205-No), the edge tracking unit 124a returns to step S4203 and repeats the same processing as described above. On the other hand, when the horizontal pixel position i at this time is to be scanned by the horizontal pixels of the captured image (S4205-Yes), the edge tracking unit 124a adds “1” to the vertical pixel position j. (S4206), the pixel position i in the horizontal direction is set to the initial value “0”, the process returns to step S4203, and the same processing as described above is repeated (S4207).

  If the pixel (i, j) is determined to be an edge (S4203-Yes), the edge tracing unit 124a further performs edge tracing in the upper right direction and the upper left direction as described below, and forms an effective edge portion forming a document margin as follows. Is specified, and the edge effective part straight line type calculation unit 126 performs a straight line detection process of the effective edge part.

  Specifically, first, after setting the linecount to “0” (S4208), the edge tracking unit 124a performs edge tracking further upward starting from the detected edge pixel (i, j). , Find a valid edge pixel forming a straight line. In this case, as described above, since the search is performed at 45 ° intervals, first, an edge straight line detection process for tracing edges in the upper right direction (S4209), and an edge straight line detection process for tracing edges in the upper right direction (S4210) The edge straight line detection processing for tracking the edge in the upper left direction (S4211) and the edge straight line detection processing for tracking the edge in the upper left direction (S4212) are performed in this order.

  After the process of step S4212 is completed, the edge tracking unit 124a finds a pixel position for restarting edge tracking according to a predetermined condition as desired (S4213-YES) to detect another edge straight line (details will be described later). ), The edge tracking start coordinate adjustment processing is performed with the edge tracking restart pixel position as a starting point (S4214). Note that the edge tracking start coordinate adjustment processing may be performed first (S4213-YES), and the processing may be restarted without performing the edge tracking start coordinate adjustment processing when an error occurs (S4213-). In addition, the operator may determine the degree of protrusion, and may set so that the edge tracking start coordinate adjustment processing is not performed only when searching from the edge side of the protruding imaging range (S4213-NO).

  As described above, the outline of the edge effective portion detection processing from the left end (S4101) has been described. From the other upper end, right end, and lower end, according to the charts shown in FIGS. 41 (B) and 41 (B). Change the content of a processing step. For example, in the case of the edge valid part detection processing S4102 from the upper end, the edge tracking unit 124a is shown in the chart of FIG. 41B from the point (i, j) = (0, pv) to the lower right. Pixels are scanned according to the processing contents of steps S4201 to S4207.

  In addition, the edge valid part detection unit 124 starts from the first edge pixel detected by the edge tracking unit 124a as a starting point and follows the tracking direction of steps S4209 to S4212 shown in the table of FIG. Edges are traced in each direction in the order of → lower right 1 → lower right 2 to identify an effective edge portion forming the edge of the document, and the edge effective portion straight line type calculating section 126 performs straight line detection processing of the effective edge portion. The description from the right end and the lower end is omitted.

  FIG. 45 is a flowchart illustrating an outline of an edge straight line detection process of performing edge tracking in a predetermined direction by the edge tracking unit 124a. Here, a description will be given of an example in the upper right direction of step S4209 shown in FIG.

  First, the edge tracking unit 124a applies the edge tracking filter shown in FIG. 42 (B) to the upper right direction, that is, (i, j + 1) and (i + 1, j + 1) for the target pixel (i, j). ) Is performed (S4500).

  The edge pixel counting unit 124b monitors the count number count for which the edge tracking unit 124a has performed the edge tracking, and when the count number count is equal to or larger than a predetermined threshold α (S4530-Yes), the edge valid unit straight line The expression calculating unit 126 performs an edge effective portion straight line expression calculating process (details will be described later) assuming that the edge pixels forming the straight line have been detected by the predetermined amount α or more (S4540). Then, an edge valid part straight line overlap determination process (details will be described later) is performed (S4560).

  Here, it is necessary to set the threshold value α depending on how long a straight line is to be recognized by the edge effective portion. Further, it is necessary to set α depending on the degree of protrusion, and it is not appropriate that α is almost equal to or smaller than ph and pv. If α is almost equal to pv and ph, the edge effective portion is hard to be detected, and it is impossible to cope with most protruding portions, which is not appropriate. Further, if α is a very small value with respect to ph and pv, there is a possibility that an edge effective portion is erroneously detected, so this is not appropriate. Here, as an example of an appropriate α, α is set to a smaller value of ph / 5 or pv / 5.

  If the number z of the linear formulas is less than four after the edge valid portion straight-line overlap determination process (S4580-No), the edge valid portion straight-line formula calculation unit 126 performs the edge valid portion straight-line formula calculation process in this direction. finish. If the number z of the linear formula is four or more (S4580-Yes), the edge valid part detection processing for any one of steps S4101, S4102, S4103, and S4104 is terminated. Since a rectangular original is handled, it is virtually impossible that the number z of the linear expression becomes 5 or more in step S4580. Therefore, the determination condition in step S4580 may be z = 4.

  FIG. 46 is a flowchart illustrating a detailed example of edge tracking processing in a predetermined direction by the edge tracking unit 124a. Here, an example of one upper right direction (S4500) shown in FIG. 45 will be described. However, as for the other seven directions, as shown in FIG. 42, only an edge tracking filter (that is, a tracking direction) to be used is different. Thus, there is no difference in the basic processing procedure.

  First, the edge-effective-portion straight-line calculation unit 126 initially sets the number of detected edges count to “0 (zero)” (S4510). Here, the edge-effective-portion straight-line calculation unit 126 takes over the processing from step S4203 shown in FIG. 44. Therefore, at the time of starting this processing, the edge-effective-part detection unit 124 uses the edge pixel as the pixel (i, j). Has been detected. Therefore, the edge-effective-part-linear-expression calculating unit 126 sets the horizontal pixel position i of the pixel (i, j) detected as an edge pixel to xs_tmp and the vertical pixel position j to ys_tmp (S4512). .

  Next, the edge-effective-part-linear-expression calculating unit 126 determines whether the pixel (i, j) is a target pixel, and (i, j + 1) immediately above the pixel (i, j + 1) is an edge pixel. (S4514). When (i, j + 1) is an edge pixel (S4514-Yes), the edge valid part linear equation calculation unit 126 adds “1” to the vertical pixel position j (S4516), and also adds “1” to the edge detection number count. 1 "is added (S4518), the process returns to step S4514, and the same process is repeated.

  On the other hand, if (i, j + 1) is not an edge pixel (S4514-No), the edge-effective-part linear-expression calculating unit 126 determines that (i + 1, j + 1), which is one pixel to the upper right of the pixel (i, j), is an edge pixel. Is determined (S4520). If (i + 1, j + 1) is an edge pixel (S4520-Yes), the edge-effective-part-linear-expression calculating unit 126 adds “1” to each of the horizontal pixel position i and the vertical pixel position j (S4522), Also, “1” is added to the number of detected edges count (S4518), and the process returns to step S4514 to repeat the same processing. The edge valid part straight line type calculation part 126 ends this processing when the edge pixel is no longer detected (S4520-No).

  In this way, the above process is repeated while sequentially changing the position of the target pixel to the next edge pixel, and the position of the target pixel (i, j) is changed to any of (i, j + 1) and (i + 1, j + 1). The above-described edge tracking process is repeated until no edge pixel is detected. By doing so, edge pixels that form a straight line within a range of directly above (90 degrees) and 45 degrees to the upper right, starting from the edge pixel first detected by the edge valid part detection unit 124, are sequentially tracked.

  FIG. 47 is a flowchart illustrating a detailed example of the edge-effective-portion straight-line calculation process (S4540) by the edge-effective-portion straight-line calculation unit 126. FIGS. 48 and 49 are diagrams illustrating a specific example of the straight line detection processing by edge tracking according to the third embodiment. Here, FIG. 48 shows a case in which a captured image in which four linear expressions of the document margin are found at the upper right corner and the upper right corner of the captured image range after the perspective transformation and four linear expressions of the document margin are found ( (At the time of detection of four sides). FIG. 49 shows an example in which two adjacent vertices of the original are protruded from the imaging range, and a captured image in which three linear expressions on the original margin are found is to be processed (at the time of detection of three sides). .

  The edge effective part straight line type calculation unit 126 starts the edge effective part straight line type calculation processing when the count number count of the edge tracking is equal to or larger than the threshold α (S4530-Yes). Here, "z" is processed as the number of the straight line type to be obtained from now on. However, at this point, it is undetermined (determined in step S4572 to be described later), and is temporarily set to be the same as the number of linear expressions obtained so far. Therefore, initially, “z = 0 (zero)”. If it does not overlap with the straight line formula calculated by the overlap determination process of FIG. 50 described later, it is assigned as a new straight line formula.

  First, the edge-effective-portion linear-expression calculating unit 126 sets xs_tmp set in step S4512 to the starting point xs (z) of the horizontal coordinate position related to the z-th linear formula, and ys_tmp to the vertical coordinate related to the z-th linear formula. The horizontal pixel position i at the start point ys (z) of the position and (i, j) when no edge is detected in step S4500 is the end point xe (z) of the horizontal coordinate position related to the z-th linear equation. ), The vertical pixel position j is set as the end point ye (z) of the vertical coordinate position related to the z-th linear equation (S4542).

  For example, as shown in FIG. 48, in the document range specification processing from the left end, if the edge is traced in the upper right direction to obtain the first straight line equation, the coordinate position of the point a is set to the start point (xs (1 ), Ys (1)), the coordinate position of the intersection d between the document margin and the right end of the imaging range is the end point (xe (1), ye (1)). After that, if the second straight line equation is obtained by performing edge tracing in the upper left two directions, the coordinate position of the point a is set to the start point (xs (2), ys (2), as in the case of the first start point. ), The coordinate position of point b is set as the end point (xe (2), ye (2)).

  Next, the edge-effective-portion straight-line-expression calculating unit 126 calculates the slope a (z) and intercept b (z) of a straight line passing through (xs (z), ys (z)) and (xe (z), ye (z)). ) Is calculated according to the equation (34) (S4544). Further, the edge effective part straight line formula calculating unit 126 calculates the straight line formula y according to the formula (35) (S4546).

  FIG. 50 is a flowchart illustrating a detailed example of the edge-effective-portion linear-type overlap determination process (S4560) by the edge-effective-portion detecting unit 124.

  When the straight line equation is obtained for the first time in the edge effective part straight line equation calculating unit 126, that is, when the number z of the straight line equations is 0 (S4562-Yes), the edge effective part detecting unit 124 sets the number z and the linecount to Is added, and the edge valid part straight line overlap determination processing is terminated (S4572).

  On the other hand, when the number z of the linear formulas is not 0 (S4562-No), the edge valid part detecting unit 124 calculates the slope a (z) and the intercept b (z) of the z-th linear formula calculated in step S4540, It is determined whether or not the slope a (k) and the intercept b (k) of the linear equation calculated so far overlap each other. Therefore, the edge valid part detecting unit 124 first sets the operator k to "1" (S4564), and determines whether a (z) and b (z) are substantially equal to a (k) and b (k), respectively. It is determined whether it is (S4568). If both are almost equal (S4568-Yes), it is determined that the straight line formula obtained this time overlaps with the already obtained straight line formula, and the edge valid part straight line type overlap determination processing is terminated (S4572).

  If any one of them is not equal (S4568-No), the edge valid part detection unit 124 adds "1" to the operator k (S4570), and repeats the determination in step S4568 while k ≦ z. If the slope a (z) and the intercept b (z) of the linear equation obtained this time are not equal to the slope a (k) and the intercept b (k) of the known linear equation, respectively, it is assumed that the linear equation is an effective linear equation. Then, "1" is added to each of the number z and the linecount (S4572), and the edge valid part straight line type overlap determination processing S4560 ends.

  FIG. 51 is a diagram for explaining the significance of the above-described edge valid part straight line overlap determination processing. Note that, in order to simplify the description, the edge tracking start coordinate adjustment processing (S4214) is not considered here.

  For example, in the document range specifying process from the left end, edge tracking is performed starting from the first edge pixel detected by the edge valid part detecting unit 124 in the order of upper right one direction → upper right two directions → upper left one direction → upper left two directions. Is used to calculate the edge straight line formula. In this case, as shown in FIG. 51A, if the document does not protrude from the imaging range, the edge-effective-part linear-expression calculating unit 126 obtains a linear expression in the following order.

  For example, the edge pixel of the point a is first detected by the edge pixel detection from the left end by the edge valid part detection unit 124 (S4101). The edge effective part straight line formula calculation unit 126 performs edge tracking in the upper right direction and the upper right direction starting from the point a (S4209, 4210). In the illustrated example, the lower left side of the document is inclined upward so that the lower long side of the document is not more than 45 degrees with respect to the lower end of the imaging range. Therefore, the straight line formula L1 is obtained by edge tracking in the upper right direction. However, a straight-line equation cannot be obtained in edge tracking in the upper right two directions.

  Next, the edge-effective-portion straight-line calculation unit 126 performs edge tracing in the upper left direction and the upper left direction starting from the point a (S4211, 4212). In the illustrated example of the inclination, a straight line formula is not obtained in the edge tracking in the upper left direction, and a straight line formula L2 is obtained in the edge tracking in the upper left direction. Here, since the slope and intercept are different between the straight line formula L1 obtained first and the straight line formula L2 obtained thereafter (S4568-No), the edge effective part straight line formula calculation unit 126 converts the straight line formula L2 into an effective straight line. As a formula, “1” is added to each of the number z and the linecount (S4572). Therefore, at this time, the effective straight lines are L1 and L2.

  After that, the edge pixel of the point b is detected by the edge pixel detection from the upper end by the edge valid part detection unit 124 (S4103). With the point b as the starting point, the edge effective part straight line formula calculation part 126 performs edge tracking in the upper right one direction and the upper right two directions in this order (S4209, 4210). In the illustrated example of the inclination, the straight line formula L3 is obtained in the edge tracking in the upper right direction, and the straight line formula is not obtained in the edge tracking in the upper right direction. Here, the straight line formula L3 obtained this time has the same slope as the already obtained straight line formula L1, but a different intercept, and also has a different slope and intercept from the straight line formula L2 (S4568-No). The straight line formula calculation unit 126 adds “1” to each of the number z and linecount using the straight line formula L3 as an effective straight line formula (S4572). Therefore, at this point, three effective straight lines are L1, L2, and L3.

  Next, the edge effective part straight line formula calculation part 126 performs edge tracing in the lower right one direction and the lower right two directions starting from the point b (S4211, 4212). In the illustrated example of the inclination, a straight line formula is not obtained in the edge tracking in the lower right direction, but a straight line formula L2a is obtained in the edge tracking in the lower right direction. Here, since the straight line formula L2a obtained this time has the same inclination and intercept as the already obtained straight line formula L2, the edge effective part straight line formula calculating unit 126 sets the straight line formula L2a as an invalid straight line formula (S4568- Yes).

  Hereinafter, similarly, the edge valid part straight line formula calculation unit 126 starts with the edge pixel of the point c detected by the edge pixel detection from the right end (S4105) as the starting point, and performs the straight line L4 (S4210), and a straight line formula L3a is obtained by edge tracing in the two lower left directions (S4212). The straight line formula L3a has the same slope and intercept as the already obtained straight line formula L3, and there is no known straight line formula L1, L2, and L3 having both the same slope and intercept as the straight line formula L4. Therefore, the edge-effective-portion straight-line calculation unit 126 sets the straight-line formula L4 as a valid straight-line formula and the straight-line formula L3a as an invalid straight-line formula (S4568). Therefore, at this time, the effective straight lines are L1, L2, L3, and L4.

  Finally, the edge-effective-part linear-expression calculating unit 126 obtains a linear expression L4a by performing edge tracing in the upper left two directions starting from the edge pixel of the point d detected by edge pixel detection from the lower end (S4107) (S4210). ), And a straight line formula L1a is obtained by edge tracing in the two lower left directions (S4212). The linear equation L4a has the same inclination and intercept as the already obtained linear equation L4, and the linear equation L1a has the same inclination and intercept as the already obtained linear equation L1. The calculation unit 126 sets any of the straight line formulas L4a and L1a obtained this time as an invalid straight line formula (S4568-Yes). As described above, the four linear formulas L1, L2, L3, and L4 forming the edge of the document can be specified.

  On the other hand, as shown in FIG. 51B, when a part of the original is out of the imaging range, the following is performed. For example, in the edge pixel detection from the left end and the upper end by the edge effective portion detecting section 124, the same starting points a and b as shown in FIG. 51A are detected. Three linear equations L1, L2, L3 are obtained in the same manner as described above for (A).

  On the other hand, as shown in FIG. 51B, since a part of the document protrudes from the upper end of the imaging range, the edge valid part detection unit 124 detects the edge pixel from the right end by detecting the edge of the document. The intersection c between the image and the upper end of the imaging range is detected as an edge pixel (S4105). The edge valid part straight line formula calculation unit 126 obtains a straight line formula L4 by performing edge tracing in the two lower right directions with the intersection c as a starting point (S4210). It is considered that a straight line formula L5 that forms the boundary of the protruding portion can be obtained by edge tracing in the lower left direction (S4211). However, since no edge pixel exists in this portion, the straight line formula L5 cannot be obtained. . Since there is no known straight line formula L1, L2, L3 in which both the slope and the intercept are the same as the straight line formula L4, the edge effective part straight line formula calculating unit 126 sets the straight line formula L4 as an effective straight line formula. (S4568-No). Therefore, at this time, the effective straight lines are L1, L2, L3, and L4.

  In addition, since a part of the document protrudes from the right end of the imaging range, the edge valid part detection unit 124 sets the intersection d between the edge of the document and the right end of the imaging range as an edge pixel in edge pixel detection from the lower end. It is detected (S4107). The edge valid part straight line formula calculation unit 126 obtains a straight line formula L1a by tracking the edge in the lower left two directions with the intersection d as a starting point (S4212). A straight line formula cannot be obtained by edge tracking in the upper left direction and the upper left direction (S4209, S4210). Since the straight line formula L1a has the same inclination and intercept as the already obtained straight line formula L1, the edge valid part straight line formula calculating unit 126 sets the straight line formula L1a obtained this time as an invalid straight line formula (S4568-Yes). As described above, the four linear formulas L1, L2, L3, and L4 forming the edge of the document can be specified. Note that the linear formulas L5 and L6 indicating the boundaries of the protruding range can be specified as being the same as the linear formulas at the edges of the imaging range.

  As described above, according to the processing procedure shown in FIG. 50, first, the edge effective portion detecting section 124 searches the inside of each of the four corners of the imaging range for the edge pixels forming the edge of the document, and finds 1 By tracing the edge pixels a predetermined distance further inward within a range of 180 degrees with one edge pixel as a starting point, a straight line formula forming the margin of the document is specified. Thus, even when a part of the document is out of the imaging range, the linear formulas forming the edges of the document can be reliably specified without overlapping.

  Further, first, the edge pixel is searched for from the corner (corner) of the imaging range toward the inside, and the edge pixel is further traced within a range of 180 degrees from the edge pixel found as a starting point. In the case of finding a straight line formula forming the margin of a document by finding a pixel and then tracing the edge, the processing can be completed in a shorter time than in the case where the edge tracing must be performed in the range of 360 degrees.

  FIG. 52 is a flowchart showing a detailed example of the edge tracking start coordinate adjustment processing (S4214). FIG. 53 is a chart showing the correspondence between the processing contents of each step of the edge tracking start coordinate adjustment processing in the processing from the top, bottom, left, and right ends.

  FIG. 54 is a diagram showing an example of a protruding state corresponding to each step in this processing. FIG. 55 and FIG. 56 are diagrams showing specific examples of the straight line formula forming the margin of the document specified by the above-described document range specifying process.

  In FIG. 52, the edge tracking start coordinate adjustment processing (S4214) in the document range identification processing from the left end (S4101, S4102) will be described. However, the processing is not limited to the left end, and this processing is performed from the top, bottom, left, and right ends. This is performed in each of the document range specifying processes. In that case, the contents of the processing steps shown in FIG. 52 may be changed as shown in FIG.

  The edge valid part detecting unit 124 first determines whether or not the linecount number is “2” (S4710). Then, when the number of linecounts is “2” (S4710-Yes), the edge valid part detecting unit 124 firstly determines the horizontal coordinate position xe (1) for the two linear expressions starting from the found edge pixel at point a. ) And xe (2) are compared (S4712). When xe (1) is smaller than (less than) xe (2), the edge valid part detector 124 sets the vertical coordinate position ye (1) to j and the horizontal coordinate position i to “0” ( S4712-Yes, S4714). On the other hand, when xe (1) is greater than (or greater than) xe (2) (S4712-No), the edge valid part detection unit 124 sets ye (2) to j and sets the horizontal coordinate position i to “0”. (S4716). If linecount is "1" (S4710-No, S4711-Yes), ye (1) is set to j, horizontal coordinate position i is set to "0" (S4718), and linecount is "0". (S4710-No, S4711-No), the vertical coordinate position j is set to "j + 1" and the horizontal coordinate position i is set to "0", so that the edge tracking start coordinate adjustment process is finally ended ( S4719).

  The significance of the determinations in steps S4710 and S4712 and the setting of the point j based on the results is to make the edge pixel on the left end as a restart location as much as possible. Except at the time of the search from the left end, the processing contents are switched in accordance with the chart of FIG. 53 so that the upper end / right end / lower end edge pixels are set as resuming places as much as possible.

  By performing the edge tracking start coordinate adjustment processing (S4214), there is a possibility that three or four edges can be detected only by searching from the left end and the upper end. In that case, the search of the right end and the lower end is unnecessary, There is an effect that the processing time can be reduced because the processing need not be performed.

  For example, in a case where a captured image that is protruding from the upper right corner and the upper right corner of the captured image area after the perspective transformation is to be processed, the edge tracking is restarted by the edge tracking start coordinate adjustment processing (S4214). The coordinates (edge tracking restart pixel position) to be set are set to (0, ye (1)) as shown in FIG.

  For example, first, in the document range specification processing (S4002, S4004), the non-inspection area specification processing unit 120 performs the first processing in the upper right direction starting from point a in the document range specification processing from the left end (S4101, S4102). , And a second linear expression y = a (2) .x + b (2) is detected in the upper left two directions.

  Thereafter, the edge valid part detecting unit 124 restarts edge tracking from (0, ye (1)). As a result, as shown in FIG. 54 (A), the third straight line equation y = a (3) · x + b (3) is detected in the upper right direction starting from point b. At this time, since only one linear expression can be detected, the j coordinate of the restart position is set to ye (1) (S4718). Thereafter, j = j + 1 and i = 0 are set (S4719), and the search is restarted again. Finally, through step S4202-Yes, the edge tracking start coordinate adjustment processing ends (S4719). This completes the search from the left end.

  In the original range specification process from the upper end (S4101, S4102), first, a straight line is detected starting from point b. At this time, two lines can be detected, but since they have already been detected, the linecount becomes "0" (S4710-No, S4711-No), and the coordinates (edge tracking restart pixel The i side of the position) is set to i + 1, and the j side is set to 0 (S4719).

  Here, in the illustrated protruding state, the captured image protrudes above the imaging range, so that if the edge tracking start coordinate adjustment processing is performed, there is a possibility that an appropriate processing may not be performed. In such a case, the operator may judge the degree of protrusion, and the edge tracking start coordinate adjustment processing may be skipped only when searching from the edge side of the protruding imaging range (S4213-NO). By doing so, the search then proceeds from the right end, and finally, as shown in FIG. 54B, y = a (4) .x + b (4) is detected in the two lower right directions. As a result, it is possible to specify all four linear formulas forming the four edges of the rectangular document. FIG. 55 shows the processing result when the four sides are detected. FIG. 55 shows not only the straight line formula forming the edge of the original document obtained as described above, but also the straight line formula forming the edge of the imaging range.

  Although the description is omitted, as shown in FIG. 54 (C), when the entire original is within the imaging range, the edge tracking start coordinate adjustment processing may be performed without any inconvenience. Can be performed.

  According to such processing, the number of straight lines is determined in step S4405. Therefore, in the case shown in FIG. All four linear expressions can be specified up to the specific processing. The processing time can be shortened accordingly.

  In addition, when a captured image in which two adjacent vertices of the original are protruded from the imaging range at the upper end of the captured image range after the perspective transformation is to be processed, the leftmost end is determined in the original range specification processing (S4002, S4004). In the document range specification process (S4101, S4102), y = a (1) .x + b (1) is detected in two upper right directions, and y = a (2) .x + b (2) is detected in one upper left direction. Further, in the edge effective straight line output process S4102 from the upper end, by detecting y = a (3) · x + b (3) in the lower right direction, four linear formulas forming four edges of the rectangular original are detected. Among them, three straight lines can be specified. FIG. 56 shows the processing result when the three sides are detected. Note that FIG. 56 also shows not only the straight line formula that forms the edge of the document obtained as described above, but also the straight line formula that forms the edge of the imaging range.

  In this case, in the case shown in FIG. 49, although the number of linear formulas is determined in step S4405, only three linear formulas are obtained, so processing is performed from all of the upper, lower, left, and right edges. The edge tracing restart pixel position set by the edge tracing start coordinate adjustment process (S4214) in the processes from "from the upper edge", "from the right edge", and "from the lower edge" is as illustrated. Here, the description of the process through which they are set will be omitted.

  FIG. 57 is a flowchart showing the outline of the procedure of the document vertex setting process by the document vertex setting processing unit 132. In the third embodiment, the processing content is changed according to the number of linear formulas indicating the edges of the document obtained in the document range specifying process as described above. For this reason, the document vertex setting processing unit 132 first determines whether or not the number (the number of straight lines) z of the linear formulas forming the document edge obtained by the edge valid part linear formula calculation unit 126 is “4” (z = 4). It is determined (S4801).

  If the number z of straight lines is “4” (S4801-Yes), the document vertex setting processing unit 132 can obtain the coordinates of the four vertices of the rectangular document by using these four linear expressions. Then, a four-point vertex setting process is performed (S4802). The protruding state to be subjected to the four-point vertex setting process is, for example, a case in which a captured image that is protruding at the upper right corner and the upper right corner of the captured image area after perspective transformation is to be processed. is there.

  On the other hand, if the number of straight lines z is not “4” (S4801-No), the document vertex setting processing unit 132 determines whether or not the number of straight lines z is “3” (z = 3) (S4803). If the number of straight lines z is “3” (S4803-Yes), the document vertex setting processing unit 132 can obtain the coordinates of the two vertices of the rectangular document by using these three linear expressions. Then, a two-point vertex setting process is performed (S4804). The protruding state to be subjected to the two-point vertex setting process is, for example, a case where a picked-up image in which two adjacent document vertices are protruding from the imaging range is set as a processing target.

  On the other hand, when the number z of straight lines is equal to or less than “2” (S4803-No), only one vertex of the document is obtained by itself, and it is difficult to set a non-inspection area. Is executed (for example, the processing is immediately stopped), error processing is performed (S4805).

  FIG. 58 is a flowchart illustrating an example of a procedure of a four-point vertex setting process by the document vertex setting processing unit 132. FIG. 59 is a diagram for explaining states to be determined in this processing and vertex calculation processings 1 to 3 corresponding thereto. In (A) to (C) of FIG. 59, z indicates the number of the straight line formula obtained by the processing by the edge effective part straight line formula calculation unit 126.

  The document vertex setting processing unit 132 checks the positional relationship between the document edges indicated by the straight line formula specified by the edge effective unit straight line formula calculation unit 126, and based on the result, orthogonal coordinates for obtaining four vertex coordinates, respectively. The order of the two straight lines 1 and 2 is specified.

  Therefore, the document vertex setting processing unit 132 first compares the inclinations a (1) and a (2) of the first linear expression (z = 1) and the second linear expression (z = 2) (S4901). . When the inclinations of the two linear equations are substantially equal and the difference | a (1) −a (2) | is smaller than the threshold value β (S4901-Yes), the document vertex setting processing unit 132 shows in FIG. As described above, it is determined that the two straight lines are linear formulas forming opposite edges of the document, and the vertex calculation process 1 based on this determination is performed (S4902).

  When a rectangular document is handled, the threshold value β may be "0 (zero)" in principle, but there are cases where the threshold value β is not actually an accurate rectangle, and the linear expression indicating the document margin is also accurately obtained. Therefore, the threshold value β is set in consideration of these points.

  If the difference | a (1) −a (2) | between the inclinations of the two linear expressions is not less than the threshold value β (S4901—No), the document vertex setting processing unit 132 then proceeds to the first linear expression (z = 1 ) Is compared with the gradients a (1) and a (3) of the third linear expression (z = 3) (S4903). When the inclinations of the two linear expressions are substantially equal and the difference | a (1) −a (3) | is less than the threshold value β (S4903-Yes), the document vertex setting processing unit 132 shows in FIG. As described above, it is determined that the two straight lines are linear formulas forming opposite edges of the document, and the vertex calculation process 2 based on this determination is performed (S4904).

  If the difference | a (1) -a (3) | between the slopes of the two linear equations is not less than the threshold value β, ie, | a (1) -a (2) | and | a (1) -a (3) | Are not less than the threshold β (S4903-No), the document vertex setting processing unit 132, as shown in FIG. 59 (C), sets the remaining fourth straight line equation (z = 4) and the first straight line It is determined that the equation (z = 4) is a straight-line equation forming the opposite edge of the document, and vertex calculation processing 3 based on this determination is performed (S4905).

  The document vertex setting processing unit 132, based on the positional relationship between the document edges indicated by the straight line equation specified in the above process, as shown in FIG. The order of each of the straight lines 1 and 2 is specified, and the intersection coordinates forming the vertex of the document are obtained based on the result.

  For example, in the vertex calculation process 1 corresponding to the state shown in FIG. 59A, the first (z = 1) linear equation and the second (z = 2) linear equation are parallel, and the first (z = 2) = 1) and the intersection of the third (z = 3) or fourth (z = 4) linear equation, and the second (z = 2) linear equation and the third (z = 3) or It is determined that the intersection with the fourth (z = 4) linear equation forms the vertex of the document.

  Then, the document vertex setting processing unit 132 calculates the first (z = 1) linear equation y = a (1) · x + b (1), which is the straight line 1 for calculating the vertex, and the straight line 2 for calculating the vertex. From the third (z = 1) linear equation y = a (3) · x + b (3), and similarly to the intersection (xc1, yc1), the first (z = 1) linear equation y = a (1) · From x + b (1) and the fourth (z = 4) linear expression y = a (4) · x + b (4), the intersection (xc2, yc2) and the second (z = 2) linear expression y = a (2) X + b (2) and the third (z = 3) linear equation y = a (3). From x + b (3), the intersection (xc3, yc3) and the second (z = 2) linear equation y = a (2) ) · X + b (2) and the intersection (xc4, yc4) from the fourth (z = 4) straight-line equation y = a (4) · x + b (4) are calculated as vertex coordinates. The table shown in FIG. 59 (D) summarizes this.

  Although detailed description is omitted, the original vertex setting processing unit 132 also performs vertex calculation processing 2 corresponding to the state illustrated in FIG. 59B and vertex calculation processing 3 corresponding to the state illustrated in FIG. As in the case of the vertex calculation process 1, the coordinates of the four vertices are calculated as shown in the table of FIG.

  FIG. 60 is a diagram illustrating a specific example of four document vertices obtained by the above-described four-point vertex setting process. The target of the four-point vertex setting process is a case where a captured image in which all four sides of the document are within the imaging range is to be processed as shown in FIGS. ) Is an example. In the state shown in FIG. 60, the first (z = 1) and third (z = 3) straight lines are straight lines forming opposite edges of the document, and thus the document vertex setting processing unit 132 performs the vertex calculation process 2 (S4904) is performed.

  Thus, the document vertex setting processing unit 132, as shown in the figure, determines the intersection (xc1, yc1) of the first and second linear expressions, and the intersection (xc2, yc2) of the first and fourth linear expressions. The intersection (xc3, yc3) of the third and second linear formulas (xc4, yc4) of the third and fourth linear formulas can be calculated as document vertices. As a result, the coordinates of the four vertices of the rectangular document are specified.

  FIG. 61 is a flowchart illustrating an example of a procedure of a two-point vertex setting process by the document vertex setting processing unit 132 when three sides are detected. FIG. 62 is a diagram for explaining states to be determined in this processing and vertex calculation processings 1 to 3 corresponding thereto. 62 (A) to 62 (C), z indicates the number of the straight line formula obtained by the processing by the edge effective part straight line formula calculation unit 126. The procedure of the two-point vertex setting process is basically different from that of the four-point vertex setting process shown in FIG. The processing procedure is almost the same.

  For example, the document vertex setting processing unit 132 checks the positional relationship between the document edges indicated by the straight line formula specified by the edge effective part straight line formula calculation unit 126, and obtains two vertex coordinates based on the result. Are specified as the order of the two straight lines 1 and 2 which are orthogonal to each other.

  Therefore, the document vertex setting processing unit 132 first compares the inclinations a (1) and a (2) of the first linear equation (z = 1) and the second linear equation (z = 2) (S5001). . When the inclinations of the two linear expressions are substantially equal and the difference | a (1) −a (2) | is smaller than the threshold value β (S5001-Yes), the document vertex setting processing unit 132 shows in FIG. As described above, it is determined that the two straight lines are linear formulas forming opposite edges of the document, and the vertex calculation processing 4 based on this determination is performed (S5002).

  If the difference | a (1) −a (2) | between the inclinations of the two linear expressions is not less than the threshold value β (S5001-No), the document vertex setting processing unit 132 then proceeds to the first linear expression (z = 1 ) And the slopes a (1) and a (3) of the third linear equation (z = 3) are compared (S5003). When the inclinations of the two linear expressions are substantially equal and the difference | a (1) −a (3) | is smaller than the threshold value β (S5003-Yes), the document vertex setting processing unit 132 shows in FIG. As described above, it is determined that the two straight lines are linear formulas forming opposite edges of the document, and the vertex calculation processing 5 based on this determination is performed (S5004).

  If the difference | a (1) -a (3) | between the slopes of the two linear equations is not less than the threshold value β, ie, | a (1) -a (2) | and | a (1) -a (3) | Are not less than the threshold value β (S5003-No), the document vertex setting processing unit 132, as shown in FIG. 62C, sets the document side opposite to the first linear equation (z = 4). It is determined that the straight line formula forming the edge has not been obtained, and the vertex calculation processing 6 based on this determination is performed (S5005).

  The document vertex setting processing unit 132, based on the positional relationship between the document edges indicated by the straight line equation specified in the above processing, as shown in FIG. The order of each of the straight lines 1 and 2 is specified, and the intersection coordinates forming the vertex of the document are obtained based on the result. For example, in the vertex calculation processing 4 corresponding to the state shown in FIG. 62A, the first (z = 1) linear equation and the second (z = 2) linear equation are parallel, and the first (z = 2) = 1) and the intersection of the third (z = 3) or fourth (z = 4) linear equation forms the vertex of the document.

  Then, the document vertex setting processing unit 132 calculates the first (z = 1) linear equation y = a (1) · x + b (1), which is the straight line 1 for calculating the vertex, and the straight line 2 for calculating the vertex. From the third (z = 1) linear equation y = a (3) · x + b (3), similarly to the intersection (xc1, yc1), and similarly the second (z = 2) linear equation y = a (2) · The intersection (xc2, yc2) is calculated as the vertex coordinates from x + b (2) and the third (z = 3) linear equation y = a (3) · x + b (3). FIG. 62 (D) illustrates this in a organized manner.

  Although detailed description is omitted, the original vertex setting processing unit 132 also performs the vertex calculation processing 5 corresponding to the state illustrated in FIG. 62B and the vertex calculation processing 6 corresponding to the state illustrated in FIG. Similarly to the case of the vertex calculation process 4, two vertex coordinates are calculated as shown in the table of FIG.

  FIG. 63 is a diagram illustrating a specific example of two document vertexes obtained by the above-described two-point vertex setting process. The target of the two-point vertex setting process is a case where a picked-up image in which three sides of the four sides of the document exist within the image pickup range as a target to be processed as shown in FIGS. This is one example. In the state shown in FIG. 63, since the straight line formula forming the document edge facing the first (z = 1) has not been found, the document vertex setting processing unit 132 performs vertex calculation processing 2 (S5005).

  As a result, the document vertex setting processing unit 132, as shown in the figure, determines the intersection (xc1, yc1) of the first and third linear expressions and the intersection (xc2, xc2) of the first and third linear expressions. yc2) can be calculated as document vertices.

  FIG. 64 is a flowchart showing an outline of the procedure of the protruding place determination processing (S4008) by the protruding place determination processing unit 134 and the protruding place vertex estimation processing (S4010) by the protruding vertex coordinate estimation processing unit 135.

  The protruding place determination processing unit 134 determines whether or not the number of straight lines z is “4”, that is, four straight lines forming the document margin have been found (z = 4) by the edge effective part straight line formula calculation unit 126. (S5101). If the number of straight lines z is "4" (S5101-Yes), the protruding location determination processing and the protruding location vertex estimation processing are immediately terminated.

  On the other hand, when the number z of straight lines is “3” (S5101-No), the protruding place determination processing unit 134 performs a two-point protruding direction determination process (S5102), and then the protruding vertex coordinate estimation processing unit 135 performs two points. A protruding place vertex estimating process is executed (S5103). In other words, in practice, these processes are performed when only three straight lines forming the margins of the document are found. When four straight lines are found, the process is immediately started when these processes are started. The process exits from the process and proceeds to a boundary portion specifying process and a non-inspection unit setting process (collectively, non-inspection region specifying process).

  FIG. 65 is a flowchart illustrating a detailed example of the procedure of the two-point protruding direction determination process (S5102) by the protruding location determination processing unit 134.

  The protruding place determination processing unit 134 first initializes an operator m (m is a positive integer including “0”) to “0 (zero)” (S5201). Then, the following processing is repeated while the operator h (h is a positive integer) is less than the number of straight lines z (S5202-Yes). Note that this processing is executed when the number of straight lines z is less than “3”, so that the processing is repeated for h = 1 and 2 in effect.

  The protruding place determination processing unit 134 first determines whether or not at1 · xs (z) + bt1 · ys (z) + ct1 = 0, that is, the start point (xs (z), ys (z) of the linear expression indicating the document margin. )) Is on the linear equation at1.x + bt1.y + ct1 = 0 indicating the left end of the imaging range (S5203).

  If the start point (xs (z), ys (z)) is on the left end linear equation (S5203-Yes), the protruding place determination processing unit 134 specifies the protruding direction and specifies the vertex corresponding to the boundary coordinates. 1 (the two are collectively referred to as a protruding place determination process 1) (S5204). In this state, two vertices of the document protrude to the left side of the captured image.

  When the start point (xs (z), ys (z)) is not on the left end linear equation (S5203-No), the protruding place determination processing unit 134 next performs at1 · xe (z) + bt1 · ye (z) + ct1 = 0, that is, whether the end point (xe (z), ye (z)) of the linear expression indicating the edge of the document is on the linear expression at1, x + bt1, y + ct1 = 0 indicating the left end of the imaging range. Is determined (S5205).

  If the end point (xe (z), ye (z)) is on the left end linear expression at1.x + bt1.y + ct1 = 0 (S5205-Yes), the protruding place determination processing unit 134 specifies the protruding direction and specifies the boundary coordinates. And a second process for specifying the vertex corresponding to (the both are collectively referred to as a protruding place determination process 2) (S5206). This state is also a state in which the two vertices of the document protrude to the left side of the captured image.

  Similarly, in the case where at2 · xs (z) + bt2 · ys (z) + ct2 = 0 is satisfied, that is, the starting point (xs (z), ys (z)) of the linear expression indicating the edge of the document is set at the right end of the imaging range. If the indicated straight line expression is at2.x + bt2.y + ct2 = 0 (S5207-Yes), the protruding place determination processing unit 134 specifies the protruding direction and specifies the vertex corresponding to the boundary coordinates (both are performed). The process is referred to as a protruding place determination process 3) (S5208). In this state, two vertices of the document protrude to the right side of the captured image.

  Further, when at2.xe (z) + bt2.ye (z) + ct2 = 0 is satisfied, that is, the end point (xe (z), ye (z)) of the linear expression indicating the edge of the original is the right end linear expression at2.x + bt2.y + ct2. If it is above 0 (S5209-Yes), the protruding place determination processing unit 134 specifies a protruding direction and specifies a vertex corresponding to the boundary coordinates (the protruding place determination processing 4 collectively includes both). Call) (S5210). This state is also a state in which two vertices of the document protrude to the right side of the captured image.

  When any of the protruding place determination processes 1 to 4 is completed, or when the above condition is not satisfied, the protruding place determination processing unit 134 determines whether or not ys (z) + ct3 = 0 is satisfied, that is, a straight line indicating the document margin. It is determined whether or not the starting point (xs (z), ys (z)) of the equation is on the linear equation y + ct3 = 0 indicating the upper end of the imaging range (S5211). When the start point (xs (z), ys (z)) is on the upper-end straight line formula (S5211-Yes), the protruding place determination processing unit 134 specifies the protruding direction and specifies a vertex corresponding to the boundary coordinates. (The two are collectively referred to as a protruding place determination process 5) (S5212). In this state, the two vertices of the document protrude above the captured image.

  Similarly, when ye (z) + ct3 = 0 is satisfied, that is, the end point (xe (z), ye (z)) of the linear expression indicating the edge of the document is placed on the linear expression y + ct3 = 0 indicating the upper end of the imaging range. If there is (S5213-Yes), the protruding place determination processing unit 134 performs a sixth process of specifying the protruding direction and specifying a vertex corresponding to the boundary coordinates (both are referred to as a protruding place determination process 6) ( S5214). This state is also a state in which two vertices of the document protrude above the captured image.

  Further, when ys (z) + ct4 = 0 is satisfied, that is, when the start point (xs (z), ys (z)) of the linear expression indicating the document margin is on the linear expression y + ct4 = 0 indicating the lower end of the imaging range. (S5215-Yes), the protruding place determination processing unit 134 performs a seventh process of specifying the protruding direction and specifying a vertex corresponding to the boundary coordinates (the two are collectively referred to as a protruding place determination process 7) (S5216). In this state, two vertices of the document protrude below the captured image.

  Finally, when ye (z) + ct4 = 0 is satisfied, that is, the end point (xe (z), ye (z)) of the linear expression indicating the document margin is on the linear expression y + ct4 = 0 indicating the lower end of the imaging range. In this case (S5217-Yes), the protruding place determination processing unit 134 performs an eighth process of specifying the protruding direction and specifying a vertex corresponding to the boundary coordinates (the two are collectively referred to as a protruding place determination process 8) (S5218). . This state is also a state in which two vertices of the document protrude below the captured image.

  FIG. 66 is a flowchart illustrating an example of the procedure of the protruding place determination process. Here, an example of the protruding place determination processing 1 is shown. FIG. 67 is a table showing the correspondence of the protruding place determination processing 1 to 8 with respect to the processing content of each step in this processing procedure.

  In the protruding place determination processing 1, the start point (xs (z), ys (z)) of the linear expression indicating the document margin is on the linear expression at1, x + bt1, y + ct1 = 0 indicating the left end of the imaging range. Is in the state of protruding to the left side. Therefore, the protruding place determination processing unit 134 first adds “1” to edgeleft (S5301), and sets the starting point (xs (z), ys (z)) at that time to (xb (m), yb (m)). Is set to (S5302).

  Next, the protruding place determination processing unit 134 calculates y = a (z) · x + b (z) calculated from the start point (xs (z), ys (z)) at this time in the edge effective part linear expression calculation processing S4540. Above, it is determined whether or not yc1 = a (z) · xc1 + b (z) holds (S5303). When the condition is satisfied (S5303-Yes), the protruding place determination processing unit 134 sets the point for estimating the vertex and the first straight line y1 to (xc1, yc1) and the straight line yc1 = a (z) · xc1 + b (z) at this time. ) (S5304). Here, the first straight line y1 is perpendicular to the straight line in the imaging range indicating the document edge passing through (xc1, yc1) and (xc2, yc2), and the document edge passing through (xc1, yc1). A straight line y = a (z) · x + b (z) that intersects the edge of the imaging range.

  On the other hand, if yc1 = a (z) · xc1 + b (z) does not hold (S5303—No), the protruding place determination processing unit 134 sets the point for estimating the vertex and the straight line y2 to (xc2, yc2). The straight line yc1 = a (z) · xc1 + b (z) is set (S5305). Here, the second straight line y2 is perpendicular to the straight line in the imaging range indicating the document edge passing through (xc1, yc1) and (xc2, yc2), and the document edge passing through (xc2, yc2). A straight line y = a (z) · x + b (z) that intersects the edge of the imaging range.

  Next, the protruding place determination processing unit 134 adds “1” to the operator m, and ends the protruding place determination processing 1 (S5204).

  Similarly, by associating the edgeleft of step S5301 and the xb (m) and yb (m) of step S5302 in the protruding place determination processing 1 as shown in FIG. 67, the same applies to the protruding place determination processings 2 to 8. Can be processed.

  FIG. 68 is a flowchart illustrating an example of the procedure of the two-point protruding location vertex estimation process (S5103) by the protruding vertex coordinate estimation processing unit 135. FIG. 69 is a table showing the correspondence of the direction in which the protruding state is estimated in this processing procedure.

  The protruding vertex coordinate estimation processing unit 135 first calculates a distance d between (xc1, yc1) and (xc2, yc2) (S5401). If the distance d is almost equal to a preset vertical pixel of the document (S5402-Yes), the protruding vertex coordinate estimation processing unit 135 sets dp to a horizontal pixel of the document and d to a vertical pixel of the document, and calculates The value of dp (the number of horizontal pixels) is calculated according to (36-1) (S5403). On the other hand, if the distance d is not equal to the vertical pixel of the document (S5402-No), the protruding vertex coordinate estimation processing unit 135 sets dp to the vertical pixel of the document and d to the horizontal pixel of the document, and calculates the formula (36) The value of dp (the number of vertical pixels) is calculated according to -2) (S5404).

  Note that these expressions can be applied in the above series of processing, not limited to the case where the entire one side is within the imaging range as shown in Case 1 on the right side of FIG. As shown in Case 2 of the drawing, the present invention can be applied to a case where only one vertex of (xc1, yc1) and (xc2, yc2) is within the imaging range.

  Next, the protruding vertex coordinate estimation processing unit 135 calculates a slope a formed by a straight line between (xc1, yc1) and (xc2, yc2) (S5405). Then, the protruding vertex coordinate estimation processing unit 135 performs the vertex estimation direction setting processing (positive) when the inclination a is equal to or more than “0 (zero)” (S5406-Yes, S5407), and when the inclination a is less than 0. , A vertex estimation direction setting process (negative) is performed (S5406-No, S5408).

  In the vertex estimation direction setting process (positive) and the vertex estimation direction setting process (negative), setting of a new vertex estimation direction from the already calculated vertices (xc1, yc1) and (xc2, yc2), for example, a If ≧ 0 and edgeleft = 2, the direction to be estimated is the negative direction of the x-axis. The other conditions are as shown in FIG.

  Next, the protruding vertex coordinate estimation processing unit 135 sets a point (xc3, yc3) at a distance dp on the first straight line y1 in the direction set in step S5407 or step S5408 from (xc1, yc1) as a vertex ( S5409). Here, the first straight line y1 is similar to step S5304 in FIG.

  Next, the protruding vertex coordinate estimation processing unit 135 sets a point (xc4, yc4) at a distance dp on the first straight line y2 in the direction set in step S5407 or step S5408 from (xc2, yc2) as a vertex. (S5410) The two-point protruding place vertex estimation processing S5103 ends. Here, the second straight line y2 is similar to step S5305 in FIG.

  FIG. 70 is a diagram illustrating a processing result example of the two-point protruding place vertex estimation processing (S5103) by the protruding vertex coordinate estimation processing unit 135. Here, as shown in FIGS. 37 and 49, a case is shown in which a captured image in which three sides of the four sides of the document are present within the imaging range is to be processed (when three sides are detected). By the above-mentioned two-point protruding place vertex estimation processing, a vertex (xc3, yc3) protruding to the upper end facing (xc1, yc1) and a vertex (xc4, yc4) protruding to the upper end facing (xc2, yc2) are obtained. Is estimated. Thus, the coordinates of the four vertices of the rectangular document are specified.

  FIG. 71 is a flowchart showing an outline of the boundary part specifying processing (S4012) by the boundary part specifying processing unit 136 and the non-inspection part setting processing (S4014) by the non-inspection area setting unit 138.

  First, the boundary part specifying processing unit 136 determines whether or not the number of straight lines z is “4”, that is, four straight lines forming the document margin have been found (z = 4) by the edge effective part straight line formula calculating unit 126. (S5501). If the number of straight lines z is “4” (S5501-Yes), the boundary part specifying processing unit 136 and the non-inspection area setting unit 138 detect the boundary part by detecting four sides and the non-inspection part setting processing (non-inspection part collectively). Inspection area specifying processing) (S5502) is performed. The protruding state which is the target of the non-inspection area specifying process by the four side detection is a case where the image to be processed is a protruding state at the upper right end and the upper right corner of the upper right corner of the captured image range after the perspective transformation. Is an example.

  On the other hand, when the number of straight lines z is “3” (S5501-No), the boundary part specifying processing unit 136 and the non-inspection area setting unit 138 detect the boundary part by three sides detection and the non-inspection part setting processing (summary). Then, a non-inspection area specifying process is executed (S5504). The protruding state to be subjected to the non-inspection area specifying process by the detection of the four sides is, for example, a case where a picked-up image in which two adjacent document vertices are protruding from the pick-up area is to be processed.

  On the other hand, when the number z of straight lines is equal to or less than “2” (S5503-No), only one document vertex is obtained, and it is difficult to set a non-inspection area. Error processing is performed in order to make the processing (for example, immediately stop the processing) (S5505).

  FIG. 72 is a flowchart showing an outline of the non-inspection area specifying process (S5502) by detecting four sides. The boundary part specification processing unit 136 and the non-inspection area setting unit 138 perform the non-inspection area specification processing 11 focusing on the first document vertex (xc1, yc1) (S5601), and then perform the second document vertex (xc2). , Yc2), a non-inspection area specifying process 12 is performed (S5602), and then a non-inspection area process 13 is performed, focusing on the third document vertex (xc3, yc3) (S5603). Then, the non-inspection area specifying processing 14 focusing on the vertex (xc4, yc4) of the original 4 is performed (S5604), and the non-inspection area specifying processing by detecting the four sides is completed.

  FIG. 73 is a flowchart illustrating an example of the procedure of the non-inspection area specifying process. Here, an example of the non-inspection area specifying process 11 (S5601) focusing on the first document vertex (xc1, yc1) is shown.

  The boundary part specifying processing unit 136 and the non-inspection area setting unit 138 determine the boundary between the document vertex determined to be out of the imaging range by the protruding place determination processing unit 134 and the straight line extending from the document vertex. The point of intersection with the edge of the imaging range is specified. At this time, the boundary part specifying processing unit 136 calculates the coordinates of both ends of the straight line including the document vertex on the extension line, calculated by the edge valid part straight line type calculation unit 126 based on the valid edge part forming the document margin. It is determined which edge of the captured image is in contact with the image, and based on the determination result, the direction in which the document protrudes is determined, thereby specifying the intersection coordinates between the document edge and the imaging range.

  For example, the boundary part specifying processing unit 136 determines whether the first document vertex (xc1, yc1) is out of the imaging range, and sets the space between (xc1, yc1) and the boundary coordinates of the captured image as the non-inspection area. Set. For this reason, first, the boundary part specifying processing unit 136 determines whether at1 · xc1 + bt1 · yc1 + ct1 <0 is satisfied, that is, whether the first document vertex (xc1, yc1) is protruding from the left end of the imaging range ( S5701).

  If this condition is satisfied (S5701-Yes), the boundary part specification processing unit 136 makes an intersection (xbc1, ybc1) between the straight line 1 for which (xc1, yc1) was calculated in step S4802 and the boundary line at1, x + bt1, y + ct1 = 0. , And the intersection (xbc2, ybc2) of the straight line 2 for which (xc1, yc1) is calculated in step S4802 and the boundary line at1, x + bt1, y + ct1 = 0 are specified as boundary coordinates.

  The non-inspection area setting unit 138 is a 3 surrounded by the first original vertex (xc1, yc1) and the two boundary coordinates (xbc1, ybc1) and (xbc2, ybc2) specified by the boundary part specification processing unit 136. The rectangular area is set as a non-inspection area (S5702).

  When the condition of step S5701 is not satisfied (S5701-No) or after step S5702, the boundary part specifying processing unit 136 determines whether or not yc1 + ct3> 0 is satisfied, that is, the first document vertex (xc1, yc1) is the imaging range. It is determined whether it protrudes from the upper end of (S5703). If this condition is satisfied (S5703-Yes), the boundary part specifying processing unit 136 determines the intersection (xbc3, ybc3) of the straight line 1 for which (xc1, yc1) was calculated in step S4802 and the boundary line yc1 + ct3 = 0, and step S4802. Then, the intersection (xbc4, ybc4) of the straight line 2 for which (xc1, yc1) is calculated and the boundary line yc1 + ct3 = 0 are specified as boundary coordinates.

  The non-inspection area setting unit 138 is a 3 surrounded by the first document vertex (xc1, yc1) and the two boundary coordinates (xbc3, ybc3) and (xbc4, ybc4) specified by the boundary part specification processing unit 136. The rectangular area is set as a non-inspection area (S5704).

  When the condition of step S5703 is not satisfied (S5703-No) or after step S5704, the boundary part specifying processing unit 136 determines whether at2 · xc1 + bt2 · yc1 + ct2> 0 is satisfied, that is, the first document vertex (xc1, yc1). ) Protrude from the right end of the imaging range (S5705).

  If this condition is satisfied (S5705-Yes), the boundary part specifying processing unit 136 intersects (xbc5, ybc5) the straight line 1 for which (xc1, yc1) was calculated in step S4802 and the boundary line at2.x + bt2.y + ct2 = 0. , And the intersection (xbc6, ybc6) of the straight line 2 for which (xc1, yc1) was calculated in step S4802 and the boundary line at2.x + bt2.y + ct2 = 0 are specified as boundary coordinates.

  The non-inspection area setting unit 138 is a 3 surrounded by the first original vertex (xc1, yc1) and the two boundary coordinates (xbc5, ybc5) and (xbc6, ybc6) specified by the boundary part specification processing unit 136. The rectangular area is set as a non-inspection area (S5706).

  Finally, when the condition of step S5705 is not satisfied (S5705-No) or after step S5706, the boundary part specifying processing unit 136 determines whether or not yc1 + ct4 <0 is satisfied, that is, the first document vertex (xc1, yc1). Is determined to protrude from the lower end of the imaging range (S5707). If this condition is satisfied (S5707-Yes), the boundary part specifying processing unit 136 determines the intersection (xbc7, ybc7) of the straight line 1 for which (xc1, yc1) was calculated in step S4802 and the boundary line yc1 + ct4 = 0, and step S4802. Then, the intersection (xbc8, ybc8) of the straight line 2 calculated with (xc1, yc1) and the boundary line yc1 + ct4 = 0 is specified as the boundary coordinates.

  The non-inspection area setting unit 138 is a 3 surrounded by the first document vertex (xc1, yc1) and the two boundary coordinates (xbc7, ybc7) and (xbc8, ybc8) specified by the boundary part specification processing unit 136. The rectangular area is set as the non-inspection area (S5708), and the non-inspection area identification processing 11 ends.

  In the case of the non-inspection area specifying processing 12 (S5602), the same processing is performed by changing (xc1, yc1) to (xc2, yc2) in the non-inspection area specifying processing 11. In the case of the non-inspection area specifying processing 13 (S5603), the same processing is performed by changing (xc1, yc1) to (xc3, yc3) in the non-inspection area specifying processing 11. In the case of the non-inspection area specifying processing 14 (S5604), the same processing is performed by changing (xc1, yc1) to (xc4, yc4) in the non-inspection area specifying processing 11.

  FIG. 74 is a diagram illustrating an example of a processing result of a non-inspection area specifying process (S5502) by detecting four sides by the boundary portion specifying processing unit 136 and the non-inspection area setting unit 138. Here, similarly to FIGS. 36 and 48, the processing results when a captured image in which all four sides of the document exist within the imaging range are to be processed (when four sides are detected).

  By the non-inspection area specifying process by the above-described four side detection, an area surrounded by three points (xc4, yc4), (xbc3, ybc3), and (xbc4, ybc4), and (xc2, yc2), (xbc5, ybc5) ) And (xbc6, ybc6) are respectively set as non-inspection areas.

  In the above description, a rectangular original is to be processed. However, when four sides of the original are present in the imaging range, the present invention can be applied to not only a rectangular shape but also an arbitrary square shape. On the other hand, when only three sides are present in the imaging range, it is necessary to guess the remaining one side based on a rectangle, and it is difficult to deal with an arbitrary square. However, although the processing becomes complicated, if it is determined in the processing of S5402 that d corresponds to which length of the side and the estimation processing is performed on the remaining side corresponding thereto, three sides are obtained. Even in the case of detection, it can be applied to any square.

  FIG. 75 is a flowchart showing an outline of the non-inspection area specifying process (S5504) by detecting three sides.

  Similar to the non-inspection area specification processing by four side detection (S5502), the boundary part specification processing unit 136 forms a document edge with respect to the document vertex determined to be out of the imaging range by the protruding place determination processing unit 134. A determination is made as to which edge of the picked-up image the coordinates of both ends of the straight line including the original vertex on the extension line, calculated by the edge valid part straight line type calculation unit 126 based on the valid edge portion, are determined. Then, the coordinates of the intersection between the document margin and the imaging range are specified by determining the direction in which the document protrudes.

  For this reason, first, the boundary part specifying processing unit 136 determines whether or not the edgeleft, edgetop, edgeright, and edgebottom are each “2”, that is, a straight line in which two adjacent document vertices all form one edge of the imaging range. It is sequentially determined whether or not it is located outside (S5801 to S5804). That is, it is the determination of the state in any of (A) to (D) of FIG.

  When edgeleft is “2” (S5801-Yes), when edgetop is “2” (S5802-Yes), or when edgeright is “2” (S5803-Yes), or when edgebottom is “ In the case of “2” (S5804-Yes), the boundary part specifying processing unit 136 determines the intersection of the first straight line y1 for which (xc3, yc3) was calculated in step S5409 with the straight line indicating the edge of the imaging range to be protruded. (Xb (1), yb (1)), and the intersection (xb (2), yb (2), yb (2)) are specified as boundary coordinates.

  Then, the non-inspection area setting unit 138 outputs the two document vertices (xc3, yc3) and (xc4, yc4) protruding from the imaging range and the two boundary coordinates (xb (1) specified by the boundary part specifying processing unit 136. ), Yb (1)) and (xb (2), yb (2)) are set as non-inspection areas (S5805), and non-inspection area specifying processing by three-side detection is performed. finish.

  On the other hand, if none of edgeleft, edgetop, edgeright, and edgebottom is “2”, the boundary part specification processing unit 136 sequentially determines combinations each of which is “1”, and the non-inspection area setting unit 138 determines the combination. A non-inspection area setting process according to the result is performed. In this determination, two adjacent vertices of the document are located on the corner side of the imaging range, one of them is located outside one straight line forming the edge of the corner, and the other vertex is located on the other side. This is a determination as to whether or not it is located outside the other straight line that forms an edge and intersects with this one straight line. That is, it is the determination of the state in any of (E) to (H) of FIG.

  For example, first, when edgeleft and edgetop are “1” (S5806-Yes), the boundary part specifying processing unit 136 determines that the two document vertices protrude to the upper left as shown in FIG. In cooperation with the non-inspection area setting unit 138, the non-inspection area setting processing 21 for the upper left corner is performed (S5807).

  If edgetop and edgeright are “1” (S5808-Yes), the boundary part specification processing unit 136 determines that the two document vertices protrude to the upper right side as shown in FIG. The non-inspection area setting process 22 for the upper right corner is performed in cooperation with the area setting unit 138 (S5809).

  When edgeright and edgebottom are “1” (S5810-Yes), the boundary part specifying processing unit 136 determines that the two document vertices protrude to the lower right side as shown in FIG. The non-inspection area setting process 23 is performed on the lower right corner in cooperation with the inspection area setting unit 138 (S5811).

  If edgebottom and edgeleft are “1” (S5812-Yes), the boundary part specifying processing unit 136 determines that the two document vertices protrude to the lower left as shown in FIG. In cooperation with the area setting unit 138, the non-inspection area setting processing 24 for the lower left corner is performed (S5813), and the non-inspection area specifying processing by detecting three sides is completed.

  FIG. 76 is a flowchart illustrating an example of a procedure of a non-inspection area setting process by detecting three sides for each corner. Here, an example of the non-inspection area setting process 21 (S5807) for the upper left corner is shown.

  FIG. 77 is a table showing the correspondence between the document vertex and two intersections for specifying the non-inspection area in this processing procedure. The non-inspection area setting processes 22 to 24 for the upper right, lower right, and lower left corners may be executed according to the correspondence shown in FIG.

  First, the boundary part specification processing unit 136 forms a straight line formula y = a (4) that forms a document margin outside the imaging range between the two document vertices (xc3, yc3) and (xc4, yc4) protruding from the imaging range. X + b (4) is calculated (S5901). Next, the boundary part specifying processing unit 136 determines the intersection (xb (1), yb (1)) between the first straight line y1 for which (xc3, yc3) was calculated in step S5409 and the straight line indicating the edge of the imaging range to be protruded. ), And the intersection (xb (3), yb (3)) of the straight line y calculated in step S5901 and the straight line indicating the edge of the protruding target imaging range are specified as the boundary coordinates (S5902).

  In addition, the boundary part specifying processing unit 136 determines the intersection (xb (2), yb (2) of the second straight line y2 for which (xc4, yc4) is calculated in step S5410 with the straight line indicating the edge of the imaging range to be projected )) And the intersection (xb (4), yb (4)) of the straight line y calculated in step S5901 and the straight line indicating the edge of the protruding target imaging range are specified as boundary coordinates (S5903). .

  Then, the non-inspection area setting unit 138 determines the first document vertex (xc3, yc3) protruding from the imaging range and the two boundary coordinates (xb (1)) specified by the boundary part specification processing unit 136 in step S5902. , Yb (1)) and (xb (3), yb (3)) are set as the non-inspection area (non-inspection unit 1) (S5904). Further, the non-inspection area setting unit 138 determines the second document vertex (xc4, yc4) protruding from the imaging range and the two boundary coordinates (xb (2)) specified by the boundary part specification processing unit 136 in step S5903. , Yb (2)) and (xb (4), yb (4)) are set as the non-inspection area (non-inspection unit 2) (S5905), and the non-inspection area setting processing is performed. 21 is ended.

  These processes form a rectangular document edge by obtaining a straight line that forms a document edge formed between two document vertices (xc3, yc3) and (xc4, yc4) that protrude from the imaging range. The non-inspection area is set after the four straight line equations are specified, which is practically equivalent to the non-inspection area specification processing by detecting four sides (S5502).

  FIGS. 78 and 79 are diagrams illustrating an example of the processing result of the non-inspection area specifying process (S5504) by the three-side detection by the boundary portion specifying processing unit 136 and the non-inspection area setting unit 138. Here, FIG. 78 shows a captured image in which two document vertices are out of the imaging range at the upper end of the captured image range after the perspective transformation, as shown in FIGS. 37B and 49. Shows a processing result in the case where is set as a processing target.

  In this state, since the edgetop is “2”, in the above-described non-inspection area specifying processing by three side detection, (xc3, yc3), (xc4, yc4), (xb A rectangular area surrounded by (1), yb (1)) and (xb (2), yb (2)) is set as a non-inspection area.

  On the other hand, FIG. 79 shows a processing result when two document vertices protrude to the upper left as shown in FIG. In this state, since edgeleft and edgetop are “1”, in the above-described non-inspection area specifying processing by three-side detection, the non-inspection area setting processing 21 (S5807) via step S5806 first performs FIG. , A third straight line y3 = a (4) .x + b (4) is specified.

  Then, as shown in FIG. 79 (B), a total of three points of the document vertex (xc3, yc3), the boundary coordinates (xb (1), yb (1)), and (xb (3), yb (3)) The triangular non-inspection section 1 surrounded by is set in the non-inspection area. Further, as shown in FIG. 79 (C), a total of three points of the document vertex (xc4, yc4), boundary coordinates (xb (2), yb (2)), (xb (4), yb (4)) The triangular non-inspection section 2 surrounded by is set in the non-inspection area.

  As described above, according to the configuration of the third embodiment, even when a plurality of vertices of the document of the captured image protrude from the captured image due to the tilt scan, there are four sides within the imaging range. By detecting the edge effective portions of the four sides, estimating the vertices and setting the non-inspection portion, the protruding portion can be excluded from the inspection target and compared with the reference image.

  When there are no four sides but three sides in the imaging range, by detecting an edge effective part of the three sides and estimating a vertex and setting a non-inspection part, a protruding part is determined. It can be excluded from the inspection target and compared with the reference image.

<< Response by computer >>
FIG. 80 illustrates an example of a hardware configuration in which the image inspection apparatus 5 of each of the above-described embodiments is configured as software using a CPU and a memory, that is, configured using an electronic computer (computer). FIG.

  The computer system 900 constituting the image inspection apparatus 5 includes a CPU 902, a ROM (Read Only Memory) 904, a RAM 906, and a communication I / F (interface) 908. The RAM 906 includes an area for storing captured image data.

  In addition, recording / reading for reading / recording data from / to a storage medium such as a memory reading unit 907, a hard disk device 914, a flexible disk (FD) drive 916, or a CD-ROM (Compact Disk ROM) drive 918. An apparatus may be provided. Image data is exchanged between the hardware via a data bus.

  The hard disk device 914, the FD drive 916, or the CD-ROM drive 918 is used, for example, for registering program data for causing the CPU 902 to perform software processing. Further, the hard disk device 914 includes an area for storing image data to be processed.

  The communication I / F 908 mediates the exchange of communication data with a communication network such as the Internet. Further, the computer system 900 includes a camera I / F unit 930 that functions as an interface (IOT controller) with the camera head 102, and an IOT controller 932 that functions as an interface with the print engine 570. The camera I / F unit 930 converts the output data from the camera head 102 from analog to digital and stores the data in the RAM 906 or the hard disk device 914.

  Note that not all processing of each functional part of the captured image processing unit 110 and the image image inspection processing unit 200 described in the above embodiment is performed by software, but a part of these functional parts is processed by a processing circuit 940 by hardware. May be provided.

  The computer system 900 having such a configuration can be the same as the basic configuration and operation of the image inspection device 5 described in the above embodiment. Further, a program for causing a computer to execute the above-described processing is distributed through a recording medium such as a CD-ROM 922. Alternatively, the program may be stored in FD 920 instead of CD-ROM 922. Further, an MO drive may be provided, and the program may be stored in the MO, or the program may be stored in another recording medium such as a nonvolatile semiconductor memory card 924 such as a flash memory.

  Further, the program may be downloaded from another server or the like via a communication network such as the Internet, acquired, or updated. As a recording medium, in addition to the FD 920 and the CD-ROM 922, an optical recording medium such as a DVD, a magnetic recording medium such as an MD, a magneto-optical recording medium such as a PD, a tape medium, a magnetic recording medium, an IC card, A semiconductor memory such as a miniature card can be used.

  The FD 920, the CD-ROM 922, or the like as an example of a recording medium can store some or all of the functions of the processing in the image inspection apparatus 5 described in the above embodiment. Therefore, it is possible to provide the following program and a storage medium storing the program. For example, a program for the reference image processing unit 210 and the image inspection unit 230, that is, software installed in the RAM 906 and the like includes the same functional units as the reference image processing unit 210 and the image inspection unit 230 described in the above embodiment. Prepare as software.

  Such software may be stored in a portable storage medium such as a CD-ROM or FD as image processing software or application software for image inspection, or distributed via a network.

  As described above, the present invention has been described using the embodiment. However, the technical scope of the present invention is not limited to the scope described in the embodiment. Various changes or improvements can be made to the above-described embodiment without departing from the spirit of the invention, and embodiments with such changes or improvements are also included in the technical scope of the present invention.

  Further, the above embodiments do not limit the invention according to the claims (claims), and all combinations of the features described in the embodiments are not necessarily essential to the means for solving the invention. Absent. The embodiments described above include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent features. Even if some components are deleted from all the components shown in the embodiment, as long as the effect is obtained, a configuration from which some components are deleted can be extracted as an invention.

  For example, in the above-described embodiment, a case where a document having a rectangular shape is handled has been described. However, even when a document having another angular shape is handled, the edge of the document is substantially similar to the above-described embodiment. To determine the positional relationship between the document margin and the boundary indicating the imaging range by determining the expression (linear expression) of the line segment to be expressed and specifying the coordinates of the intersection of the line and the boundary indicating the imaging range. The protruding portion may be set as the non-inspection portion.

  In addition to a document having a square shape, for example, a document having a circular shape (including an elliptical shape) or any other shape, a line segment indicating the edge of the document in the captured image is obtained. Any relationship may be used as long as the positional relationship between the line segment and the boundary line indicating the imaging range is specified, and based on the specified positional relationship, the document portion protruding from the imaging range is set as the non-inspection area. In this case, for example, by specifying the positional relationship between the document edge and the boundary line indicating the imaging range by pattern matching between the document shape known in advance and the line segment indicating the edge of the document extracted from the captured image. The portion located outside the imaging range of the document may be set as the non-inspection portion.

FIG. 1 is a schematic diagram illustrating an image inspection apparatus provided with an embodiment of an image processing apparatus according to the present invention. FIG. 2 is a block diagram illustrating a detailed example of a first embodiment of the image inspection apparatus illustrated in FIG. 1. FIG. 4 is a diagram illustrating a relationship between a captured image and a reference image. It is a block diagram showing an example of 1 composition of a non-inspection area specific processing part. FIG. 4 is a diagram illustrating a state in which a document portion to be processed in the embodiment is imaged out of an imaging range and is imaged. 5 is a flowchart illustrating an outline of a processing procedure in the image inspection apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example of an image handled in the first embodiment. FIG. 5 is a diagram illustrating another example of an image handled in the first embodiment. FIG. 9 is a diagram for describing processing of an edge extraction unit. 5 is a flowchart illustrating an outline of a procedure of a non-inspection area specifying process of a non-inspection area specifying processing unit according to the first embodiment. It is a flowchart which shows the edge extraction processing in the non-inspection area | region identification processing of the 1st example. It is a flowchart which shows the detailed example of the left side edge extraction processing (S210L of FIG. 11) in the non-inspection area | region identification processing of the 1st example. FIG. 9 is a diagram illustrating a case where the left side edge effective portion detection processing is applied to the image of FIG. It is a flowchart which shows the detailed example of the left side edge effective part detection process in the non-inspection area | region identification processing of the 1st example. 9 is a table showing determination conditions used in the edge effective portion detection processing. 15 is a flowchart illustrating a detailed example of a left-side edge valid portion provisional detection process (S310L in FIG. 14) in the non-inspection region specifying process of the first example. FIG. 17 is a flowchart illustrating a detailed example of a left-side edge valid portion selection process (S3114 in FIG. 16) in the left-side edge valid portion temporary detection process (S310L in FIG. 14) of the first example. 15 is a flowchart illustrating a detailed example of a left-side edge valid part detection process (S334 in FIG. 14) in the non-inspection area specifying process of the first example. FIG. 19 is a diagram illustrating derivation of a calculation formula used in FIG. 18. It is a flowchart which shows the detailed example of the edge effective part linear formula calculation processing (S400 of FIG. 10) by the edge effective part linear formula calculation part of 1st Embodiment. 11 is a flowchart illustrating a detailed example of a document vertex setting process (S420 in FIG. 10) by a document vertex setting processing unit according to the first embodiment. 11 is a flowchart illustrating a detailed example of a protruding place determination process (S500 in FIG. 10) by a protruding place determination processing unit according to the first embodiment. 11 is a flowchart illustrating a detailed example of a non-inspection portion setting process of a protruding portion (S600 in FIG. 10) performed by a boundary portion specifying processing unit. It is a figure which shows the specific example of the non-inspection part setting process of a protruding part. 24 is a chart showing a correspondence relationship in a case where the equations and intersection coordinates used in the processing steps for the lower left shown in FIG. 24 are applied to the upper left, upper right, and lower right. FIG. 4 is a block diagram illustrating a detailed example of a second embodiment of the image inspection apparatus illustrated in FIG. 1. This It is a flow chart which shows an outline of a processing procedure in an image inspection device of a 2nd embodiment. It is a figure showing an example of an image handled by a 2nd embodiment. It is a flow chart which shows the example of the outside place judging processing by the outside place judgment processing part of a 2nd embodiment. It is a figure showing an example in the protruding place judgment processing by the protruding place judgment processing part. It is a flowchart which shows the specific example of the non-inspection part setting process of the protruding part by the boundary part identification processing part of 2nd Embodiment. FIG. 32 is a table showing a correspondence relationship in a case where equations and intersection coordinates used in processing steps for the lower left shown in FIG. 31 are applied to the upper left, upper right, and lower right. FIG. 11 is a diagram illustrating an outline of a document range specifying process in a third embodiment of the image inspection apparatus 5 illustrated in FIG. 1. It is a figure explaining the outline of the non-inspection area specification processing in a 3rd embodiment. It is a figure explaining the outline of the processing procedure for realizing the processing contents of a 3rd embodiment. 13 is an example of a captured image showing a protruding state of a document that can be handled by the processing of the third embodiment. 13 is an example of a captured image showing a protruding state of a document that can be handled by the processing of the third embodiment. 13 is an example of a captured image showing a protruding state of a document that cannot be handled by the processing of the third embodiment. It is a block diagram showing an example of 1 composition of a non-inspection area specific processing part of a 3rd embodiment. It is a flow chart which shows an outline of a processing procedure in a non-inspection area specific processing part of a 3rd embodiment. It is a figure explaining correspondence in processing of an edge effective part primary detecting element of a 3rd embodiment. FIG. 3 is a diagram illustrating each direction of edge tracking by an edge tracking unit of an edge valid portion detection unit and an edge tracking filter. It is a flow chart which shows the outline of the procedure of edge effective part detection processing and edge effective part straight line type calculation processing in a 3rd embodiment. 9 is a flowchart illustrating a detailed example of a procedure of a document range specifying process (an edge effective portion detecting process + an edge straight line calculating process). 11 is a flowchart illustrating an outline of an edge straight line detection process of performing edge tracking in a predetermined direction by an edge tracking unit. 9 is a flowchart illustrating a detailed example of edge tracking processing in a predetermined direction by an edge tracking unit. This It is a flowchart which shows the detailed example of an edge effective part straight line type | formula calculation process (S4540). It is a figure explaining the example of the straight line detection processing by edge tracing of a 3rd embodiment (at the time of 4 side detection). It is a figure explaining the example of the straight line detection processing by edge pursuit of a 3rd embodiment (at the time of 3 side detection). It is a flow chart which shows the detailed example of edge effective part straight line type overlap judgment processing (S4560). It is a figure explaining the significance of edge effective part straight line type overlap judgment processing. It is a flowchart which shows the detailed example of an edge tracking start coordinate adjustment process (S4214). 9 is a table showing the correspondence between the processing contents of each step of the edge tracking start coordinate adjustment processing in the processing from the top, bottom, left, and right ends. It is a figure showing an example of a protruding state corresponding to each step in this processing. FIG. 9 is a diagram illustrating a specific example of a straight line formula forming an edge of a document specified by a document range specifying process (when four sides are detected). FIG. 9 is a diagram illustrating a specific example of a straight line formula forming a document edge, which is specified by the document range specifying process (when three sides are detected). 9 is a flowchart illustrating an outline of a procedure of a document vertex setting process. It is the flowchart which showed an example of the procedure of the four-point vertex setting process. It is a figure explaining the state used as a judgment object in four point vertex setting processing, and corresponding vertex calculation processing 1-3. FIG. 9 is a diagram illustrating a specific example of four document vertices obtained by a four-point vertex setting process. It is the flowchart which showed an example of the procedure of a two-point vertex setting process. It is a figure explaining the state used as a judgment object in two-point vertex setting processing, and vertex calculation processing 1-3 corresponding to it. FIG. 9 is a diagram illustrating a specific example of two document vertices obtained by a two-point vertex setting process. It is a flowchart showing the outline of the procedure of the protruding place determination processing (S4008) and the protruding place vertex estimation processing (S4010). It is the flowchart which showed the detailed example of the procedure of two point protruding direction determination processing (S5102). It is a flowchart which shows an example of the procedure of the protruding place determination processing. It is the chart which showed the correspondence of the protruding place judgment processing 1-8 about the process content of each step in the protruding place judgment processing. It is a flowchart which shows an example of the procedure of two point protruding place vertex estimation processing (S5103). It is a chart showing correspondence of the direction for estimating the protruding state in the two-point protruding place vertex estimation processing procedure. It is a figure which shows the example of the processing result by two points protruding place vertex estimation processing (S5103). It is a flowchart which shows the outline | summary of a boundary part specific process (S4012) and a non-inspection part setting process (S4014). It is a flowchart which shows the outline | summary of the non-inspection area | region identification process by four side detection (S5502). It is the flowchart which showed an example of the procedure of the non-inspection area | region identification processing. FIG. 13 is a diagram illustrating an example of a processing result of a non-inspection area specifying process (S5502) by detecting four sides by a boundary portion specifying processing unit 136 and a non-inspection area setting unit 138. It is a flowchart which shows the outline | summary of the non-inspection area | region identification process by three side detection (S5504). It is a flowchart which shows an example of the procedure of the non-inspection area setting process by three side detection about each corner. 9 is a table showing the correspondence between the document vertex and two intersections for specifying the non-inspection area in the non-inspection area setting processing procedure based on three side detection. It is a figure which shows the example of the processing result by the non-inspection area | region identification processing by three side detection (S5504) (the 1; S5805). It is a figure which shows the example of a processing result by the non-inspection area | region identification processing by three side detection (S5504) (the 2; S5807). FIG. 3 is a diagram illustrating an example of a hardware configuration when an image inspection apparatus is configured as software using a CPU and a memory.

Explanation of reference numerals

  DESCRIPTION OF SYMBOLS 1 ... System, 5 ... Image inspection apparatus, 100 ... Image reading part, 102 ... Camera head, 104 ... Shooting lens, 106 ... CCD imaging element, 110 ... Shooting image processing part, 112 ... Shading correction part, 114 ... Optical distortion correction Unit, 116: perspective transformation unit, 120: non-inspection area specifying processing unit, 122: edge extraction unit, 124: edge effective unit detection unit, 124a: edge tracking unit, 124b: edge pixel counting unit, 126: edge effective unit straight line Expression calculation section (line segment calculation section), 130: non-inspection area specifying section, 132: original vertex setting processing section (positional relation specifying processing section), 134: protruding place determination processing section, 135: protruding vertex coordinate estimation processing section, 136: boundary part specifying processing unit, 138: non-inspection area setting unit, 200: image inspection processing unit, 210: reference image processing unit, 212: reverse perspective transformation unit, 14: Resolution conversion unit, 216: Shading processing unit, 218: Non-inspection area deletion processing unit, 230: Image inspection unit, 232: Alignment processing unit, 234: Defect abnormality detection processing unit, 300: Data storage unit 320: control unit, 500: printer

Claims (13)

An image inspection method for inspecting the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
Find a line segment indicating the edge of the original in the captured image,
Identify the positional relationship between the obtained line segment and the boundary line indicating the imaging range,
Based on the specified positional relationship, a document portion protruding from the imaging range is set as a non-inspection area,
An image inspection method, wherein an image of the document portion is inspected by comparing the reference image with an image of the document portion in the captured image while excluding the non-inspection region from an inspection target. An image inspection apparatus that inspects the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
An imaging unit that captures the original and acquires a captured image;
A line segment calculation unit that obtains a line segment indicating an edge of the document in the captured image acquired by the imaging unit;
A positional relationship identification processing unit that identifies the positional relationship between the line segment obtained by the line segment calculation unit and the boundary line indicating the imaging range,
A non-inspection area setting unit that sets a document portion that protrudes from the imaging range based on the positional relationship specified by the positional relationship identification processing unit as a non-inspection area;
While excluding the non-inspection area set by the non-inspection area setting unit from the inspection target, an image comparison unit that compares the reference image and the image of the document portion in the captured image,
An image inspection device, comprising: an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit. The document has a substantially square shape,
The line segment calculation unit, obtained by the imaging unit, to obtain a straight line indicating the document edge that exists within the imaging range among the edges of the substantially angular document,
The positional relationship specifying processing unit specifies, for the straight line obtained by the line segment calculating unit, coordinates of intersections of a predetermined two straight lines to specify coordinates of vertexes of the substantially rectangular document, When the number of the straight lines determined by the line segment calculating unit is smaller than the number of edges of the substantially square document, the straight line determined by the line segment calculating unit and a reference corresponding to the substantially square document. Based on the ratio between the respective edges of the image, specify the coordinates of the vertex of the document of the substantially angular shape existing outside the imaging range,
The non-inspection area setting unit sets, as a non-inspection area, a portion of the document that protrudes from an imaging range based on coordinates of vertexes of the document specified by the positional relationship specifying processing unit. 3. The image inspection apparatus according to 2. The document has a substantially square shape,
The line segment calculation unit is a linear expression calculation unit that obtains a linear expression indicating an edge of the substantially angular document acquired by the imaging unit,
The said positional relationship specific | specification processing part is a boundary part specific | specification processing part which specifies the intersection coordinate of the boundary line which shows the imaging range with the linear formula calculated | required by the said linear formula calculation part. Image inspection equipment. A document vertex setting processing unit that specifies the vertex coordinates of the document in the captured image with reference to the linear formula obtained by the linear formula calculation unit,
Among the vertex coordinates specified by the document vertex setting processing unit, a protruding place determination processing unit that specifies vertex coordinates that protrude from the imaging range,
The image inspection apparatus according to claim 4, wherein the non-inspection area setting unit sets the non-inspection area with reference to the vertex coordinates specified by the protruding place determination processing unit. A document vertex setting processing unit that specifies a document vertex coordinate existing in the imaging range with reference to a linear formula indicating a document edge existing in the imaging range obtained by the linear formula calculation unit,
The coordinates of the vertex of the document existing outside the image capturing range based on the document vertex in the image capturing range obtained by the document vertex setting processing unit and the ratio between the respective edges of the reference image corresponding to the substantially rectangular document. And a protruding vertex coordinate estimation processing unit for estimating
The non-inspection area setting unit refers to the coordinates of the document vertices in the image capturing range obtained by the document vertex setting processing unit and the document vertices in the image capturing range obtained by the protruding vertex coordinate estimation processing unit. The image inspection apparatus according to claim 4, wherein the non-inspection area is set. An edge extraction unit that extracts an edge portion from the captured image,
An edge effective part detection unit that specifies an effective component in the edge part extracted by the edge extraction unit,
The image inspection apparatus according to claim 3, wherein the straight-line-expression calculating unit obtains the straight-line expression with reference to an effective edge portion specified by the edge-effective-portion detecting unit. The edge effective portion detection unit,
For an edge portion extracted by the edge extraction unit, an edge tracking unit that searches for edge pixels in a predetermined direction, and an edge pixel counting unit that counts the number of edge pixels searched by the edge tracking unit,
In the process of the edge tracking unit searching for the edge portion extracted by the edge extraction unit in a substantially straight line, when the edge pixel counting unit counts a predetermined number or more of the edge pixels, the edge pixels are regarded as valid. The image inspection device according to claim 7, wherein the image inspection device detects the image as a sharp edge portion. The protruding place determination processing unit sets the intersection coordinates of the straight line obtained by the straight line type calculation unit with reference to the valid edge part specified by the edge valid part detection unit as the vertex coordinates of the document. The image inspection apparatus according to claim 7, wherein: The boundary portion identification processing section may determine which edge of the effective edge portion detected by the edge effective portion detection section is in contact with any one of a plurality of boundary lines of the substantially rectangular imaging range. The image inspection apparatus according to claim 9, wherein a direction in which the document protrudes is determined to specify coordinates of an intersection with the boundary line. The protruding place determination processing unit compares the vertex coordinates specified by the document vertex setting processing unit with the boundary line indicating the imaging range, specifies the positional relationship of the protruding vertex with respect to the imaging range, and The image inspection apparatus according to claim 5, wherein it is determined in which direction the document protrudes. The image inspection apparatus according to claim 7, wherein the non-inspection area setting unit sets the non-inspection area based on a protruding direction of the document determined by the protruding place determination processing unit. A program for inspecting the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection,
Computer
A line segment calculation unit that obtains a line segment indicating an edge of the document in the captured image,
A positional relationship identification processing unit that identifies the positional relationship between the line segment obtained by the line segment calculation unit and the boundary line indicating the imaging range,
A non-inspection area setting unit configured to set a document portion protruding from an imaging range as a non-inspection area based on the positional relationship identified by the positional relationship identification processing unit;
While excluding the non-inspection area set by the non-inspection area setting unit from the inspection target, an image comparison unit that compares the reference image and the image of the document portion in the captured image,
A program that functions as an image inspection unit that inspects an image of the document portion based on a comparison result by the image comparison unit.

Images