JP2005316550A - Image processor, image reader, image inspection device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image inspection device which inspects defects of a picked-up image by comparing the picked-up image with a reference image, and has a mechanism which is not affected by three-dimensional distortion such as warpage or flexion when occurring in a read object. <P>SOLUTION: An image reading part 100 photographs a document obliquely downward to acquire a picked-up image. A tangent inclination extracting part 254 extracts the inclination of a tangent along a curve showing the contour of the document. A curled distortion region specifying part 256 compares the inclination of the tangential line extracted by the tangential line inclination extracting part 254 with a preset threshold value to specify a region having a three-dimensional distortion component. A sight-through conversion part 226 gives rectangular sight-through conversion to a trapezoidal image picked up by the image reading part 100. An image inspection part 330 inspects the picked-up image by comparing an image in an effective inspection region as a portion of the picked-up image acquired by the image reading part 100, excluding a region having the three-dimensional distortion component specified by an inspected object region specifying part 250, with the reference image. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus that performs predetermined image processing on an image obtained by optically reading document information such as characters, figures, or images described on a medium such as paper, and image reading that optically reads the document information. The present invention relates to an apparatus, an apparatus for inspecting an image by comparing an image read using the mechanism of the image reading apparatus with a reference image, and a program suitable for realizing the apparatus by software using an electronic computer. .

  More specifically, the present invention relates to a coping technique when an image is read in a state where there is a three-dimensional distortion such as warping or bending.

  An image reading apparatus that reads an image of a document, that is, a so-called scanner apparatus is used in various fields. For example, it may be used alone or may be used by being incorporated in a copying machine. Alternatively, as described in Patent Document 1, the image may be used in an image inspection apparatus that inspects an image defect or the like by comparing a read image with a reference image for inspection.

  As an image reading apparatus, for example, an original is placed on a platen glass serving as a document placement table, and a flatbed image scanner that is scanned by a line sensor to scan the document, and a user scans the image sensor by hand. There are direct image scanners that read an image by directly touching a document, such as a handy image scanner or a sheet scanner.

  There is also an indirect image scanner (also referred to as a non-contact image scanner) that indirectly reads a document by using a two-dimensional imaging device (area sensor) with a certain distance between the document and the imaging device. is there.

  The direct type scans an image while the document is pressed against the platen glass or line sensor, so the document can always be scanned with the document facing the image sensor. There is a problem that the reading speed is slow because scanning is performed.

  On the other hand, the non-contact type uses a two-dimensional area sensor that has few size and flatness restrictions on the object to be read, so there is no need to scan for reading, so the size and speed can be increased. There is an advantage that a reading device can be realized.

  As an example of a non-contact type image scanner, for example, Patent Document 2 discloses an apparatus having a function of reading an original from an obliquely upper side using an area sensor and converting the read image into an image taken from the front. .

  However, in the case of non-contact scanning, there is no mechanism for pressing the document against the platen glass or sensor. On the other hand, when read in a non-contact manner, a paper medium such as a document to be read is not flat and may actually have a three-dimensional distortion such as unevenness due to warping or bending. For this reason, even if an image read from oblique information is simply perspective-transformed and converted to an image from the front viewpoint, there is a problem that image distortion due to warping or bending of the paper remains.

  For this reason, when an image such as an image defect is inspected by comparing the image read by the non-contact type image scanner with the reference image for inspection, there is a problem that the defect detection accuracy in the distorted portion is lowered.

  As a technique for solving the problem of three-dimensional distortion in the non-contact type, Patent Document 3 discloses that a contour is extracted from document information read by a camera, vertex information is extracted from the contour information, and vertex information is known. Has disclosed a mechanism for measuring the distance between the camera and the document from the document shape information and correcting the folded document information on the plane from the distance information and the vertex information.

  In the mechanism of Patent Document 3, a document image is divided into triangular patches by connecting vertices on one side of the document image and vertices on a side facing the document image. Then, the 3D coordinates of each vertex are determined from the vertex information and distance information, and each triangle patch is perspectively transformed on the same plane based on the 3D coordinates of these vertices. Create a blank image.

JP 2000-123176 A JP 2000-022869 A JP 2002-165083 A

  However, the method of Patent Document 3 measures the distance from the document to the imaging surface (for example, a camera) of the imaging device and corrects the location dependence of the geometric distortion by performing a predetermined calculation. A mechanism is required, and the distance calculation and the correction coefficient calculation process are complicated.

  Therefore, when the image reading apparatus using the method of Patent Document 3 is applied to an image defect inspection apparatus, the apparatus configuration and calculation processing are complicated.

  The present invention has been made in view of the above circumstances, and even when a three-dimensional distortion such as warping or bending is generated on an object to be read, it is affected by taking a simple method. The purpose is to provide a mechanism without any problem.

  An image processing apparatus according to the present invention is an image processing apparatus that performs predetermined image processing on an optically read original image. An original of a captured image obtained by photographing the original from obliquely above is obtained. By analyzing the contour to be shown, an effective area specifying unit for specifying an effective area in which an area having a three-dimensional distortion component included in the document in the captured image is excluded from the processing target is provided.

  The image reading apparatus according to the present invention includes an image capturing unit that captures a captured image by photographing a document from obliquely above, and a function similar to that of the image processing apparatus according to the present invention.

  The image reading apparatus according to the present invention has the same function as that of the image reading apparatus according to the present invention, and of the identified effective area (in particular, the effective inspection area here) among the captured images acquired by the imaging unit. An image inspection unit that inspects the captured image acquired by the imaging unit by comparing the image and the reference image is provided.

  The invention described in the dependent claims defines further advantageous specific examples of the image processing apparatus, the image reading apparatus, and the image inspection apparatus according to the present invention. Furthermore, the program according to the present invention is suitable for realizing the image processing apparatus, the image reading apparatus, and the image inspection apparatus according to the present invention by software using an electronic computer. The program may be provided by being stored in a computer-readable storage medium, or may be distributed via wired or wireless communication means.

  For example, the effective area specifying unit, the extraction area specifying unit having the same function as the effective area specifying unit, and the inspection target area specifying unit extract a tangent inclination along a curve indicating the outline of the document. And a distorted area specifying unit that specifies an area having a three-dimensional distortion component by comparing the tangential inclination extracted by the tangential inclination extracting unit with a predetermined threshold value.

  According to the present invention, a three-dimensional distortion component in which warpage or deflection of a document in the captured image is generated by analyzing a contour of the captured image obtained by photographing the document from obliquely above. The effective area is specified by excluding the area that has the.

  For this reason, by setting only the effective area as the processing target, for example, the output read image does not include the three-dimensional distortion component of the document. Further, when performing image defect detection, an area having a three-dimensional distortion component included in a document can be excluded from the inspection target range, and as a result, detection accuracy can be ensured.

  In addition, since the three-dimensional distortion component of the original is not corrected but only excluded from the effective range, the processing is simple and a compact apparatus can be realized.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<First Embodiment>
FIG. 1 is a block diagram showing a first embodiment of an image processing system including an image reading apparatus equipped with an image processing apparatus according to the present invention. The image processing system 1 includes an imaging unit 10 that captures a captured image by photographing a document G from obliquely above, and an image processing apparatus that performs predetermined image processing on an image of the document optically read by the imaging unit 10. An image reading device 3 having an image processing unit 2 according to an embodiment and an image output terminal (IOT) 7 that outputs an image read by the image reading device 3 are provided. For example, by integrally configuring the image reading device 3 and the image output terminal 7, the image processing system 1 can function as a copying device.

  The image output terminal 7 is operable with various functions of the image reading device 3, and is one of raster output scan (ROS) based image forming apparatuses for operating the image processing system 1 as a digital printing system. A print engine 70 according to the embodiment, a display device 80 as an embodiment of an image display device for operating as an image display system, and the like are provided.

  The print engine 70 has a function of an interface (IOT controller) with the image reading device 3 and print output processing for performing predetermined processing for print output on the image data D10 output from the image reading device 3. 72, a laser light source 74 that emits a light beam, a laser drive unit 76 that controls or modulates the laser light source 74 according to data output from the print output processing unit 72, and a light beam emitted from the laser light source 74 is photosensitive. A polygon mirror (rotating polygonal mirror) 78 that reflects toward the member 79;

  The print output processing unit 72 generates and renders (develops into raster data) data representing a plurality of, preferably at least three separation colors, for the image data D10 according to a known technique. For example, at least three (preferably four) YCrCb color systems represented by the data D10 are mapped to, for example, the CMY color system or the CMYK color system, and color-separated raster data for print output is generated.

  Further, the print output processing unit 72, in such raster data conversion processing, undercolor removal (UCR) that reduces the CMY component of the color image, or a gray that partially replaces the reduced CMY component with the K component. Perform component exchange (GCR). Further, the print output processing unit 72 may perform color separation linearization or similar processing in order to adjust a toner image of an output image created in response to output data (CMYK or the like).

  With this configuration, the print engine 70 reflects the light beam generated by the laser light source 74 on a plurality of surfaces on the polygon mirror 78 to expose the photosensitive member 79, and forms a latent image on the photosensitive member 79 by scanning scanning. Form. When the latent image is formed, the image is developed according to an arbitrary method among many methods known in the art, and the color image read by the image reading device 3 is output as a visible image.

  When a color image is output as a visible image, the data representing the image includes at least three (preferably four) color separation data (for example, C, M, Yel, K, etc.), and each color is separated. As an image plane or in luminance-chrominance format.

  The display device 80 has a function of an interface (IOT controller) with the image reading device 3, a display output processing unit 82 that performs a predetermined output process according to the image data D10 output from the image reading device 3, and a display A display unit 84 such as a CRT, a liquid crystal (LCD), or an organic EL that displays and outputs a visible image based on the data output from the output processing unit 82 is provided. The display output processing unit 82 may be incorporated in the personal computer main body 86 used together with the display unit 84 by software or hardware, for example.

  The display output processing unit 82 generates and renders (develops into raster data) data representing a plurality of, preferably at least three separated colors, in accordance with a well-known technique, with respect to the image data D10 input from the image reading device 3. For example, mapping from the YCrCb color system represented by the image data D10 to, for example, the RGB color system is performed, and raster data that is color-separated for display output is generated. Further, the display output processing unit 82 may perform color correction processing according to the preference of the operator in such raster data conversion processing. With this configuration, the display device 80 outputs the color image read by the image reading device 3 as a visible image.

<Outline of image reading apparatus>
FIG. 2 is a block diagram showing details of an embodiment of the image reading apparatus 3 shown in FIG. The image reading device 3 captures an image from an oblique upper side to acquire a captured image, and shading correction, optical distortion correction, geometric shape correction, or document area for the captured image acquired by the imaging unit 10 A captured image processing unit 20 that performs processing such as clipping and a data storage unit 30 that stores various data are provided. The captured image processing unit 20 and the data storage unit 30 constitute the image processing unit 2 shown in FIG.

  The imaging unit 10 includes a camera head 12, a support member 14 that supports the camera head 12, and a document placement table 16 on which the document G is placed. The support member 14 is disposed in the vicinity of the document placing table 16. Alternatively, the support member 14 may be configured integrally with the document placing table 16. The support member 14 is configured such that the height of the camera head 12 relative to the document placement table 16 can be adjusted.

  The camera head 12 is installed on the support member 14 so that the orientation of the photographic lens 12b with respect to the document G placed on the document placement table 16 can be adjusted so that the document G to be read is read obliquely from above. The document table 16 is colored so that the document G and the background can be distinguished.

  The camera head 12 processes a CCD image pickup device 12a, which is an example of an area sensor for color image pickup, and image pickup signals R, G, B of each color output from the CCD image pickup device 12a, and an A / D (not shown). A reading signal processing unit (CCD camera interface circuit) 18 for converting into digital image data by a conversion circuit is included therein.

  In addition, the camera head 12 includes a photographing lens 12b that captures an image of the document G onto an imaging surface (imaging surface) of the CCD image sensor 12a. The image data output from the read signal processing unit 18 is temporarily stored in the data storage unit 30.

  The captured image processing unit 20 performs a shading correction on the digital image data output from the read signal processing unit 18 to correct pixel sensitivity variation of the CCD image pickup device 12a and to correct the light quantity distribution characteristic of the optical system (shading correction). And an optical distortion correction unit 24 that corrects distortion aberration mainly caused by the photographing lens 12b with respect to the captured image subjected to the shading correction.

  The captured image processing unit 20 includes an extraction area specifying unit 25 that specifies that an area excluding the curl distortion component is an effective extraction area in the image subjected to distortion correction, and the curl distortion component is removed. A perspective conversion unit 26 that performs processing for correcting geometric distortion of the image (perspective conversion processing), and region extraction for cutting out a necessary portion (for example, a region of the document G) from the image subjected to the geometric correction processing Part 28.

  The extraction area specifying unit 25 analyzes the outline of the captured image obtained by photographing the document from obliquely above, so that the region having the three-dimensional distortion component of the document in the captured image is processed. By performing the process of excluding (hereinafter also referred to as a distortion area excluding process), it has a function as an effective area specifying unit that specifies an effective area excluding an area having a large three-dimensional distortion component.

  The shading correction unit 22 shoots a blank sheet in advance and stores it as a white reference DW (i, j), for example, as shading correction for correcting illuminance unevenness in the imaging region and characteristic variation of each pixel of the CCD image sensor 12a. . Next, the pixel value is corrected based on the following equation (1) for the photographed image D (i, j).

  Here, P (i, j) is a corrected pixel value, i is a pixel position in the main scanning direction, and j is a pixel position in the sub-scanning direction. N is the bit resolution after correction, and if 8-bit resolution, n = 8, and P (i, j) takes values from 0 to 255 gradations.

  In the above example, the method of performing the shading correction by holding the white reference data for all the pixels has been described. However, for example, a method of setting the peak value of the entire captured image as the white reference DW, A method using the peak value DW (j) for each as a white reference can also be applied.

  The optical distortion correction unit 24 corrects the aberration caused by the photographing lens 12b as follows. For example, if the aberration at the incident angle θ to the photographing lens 12b is d, the distance from the photographing lens 12b to the CCD imaging element 12a imaging plane is c, and the distance from the optical axis of the imaging position on the imaging plane is r. The aberration d is expressed by equation (2). Therefore, the optical distortion correction unit 24 corrects the distortion based on this characteristic.

  Since the aberration d is generally proportional to the cube of r, the distortion aberration can be corrected by obtaining a proportionality constant based on the lens characteristics.

  The extraction area specifying unit 25 has an image distortion component (referred to as a curl component) due to warpage or deflection of the paper surface when unevenness due to warpage or deflection occurs on the paper surface (document) to be read. Is excluded from the inspection target, so that only an image of a necessary area is extracted.

  For this reason, the extraction area specifying unit 25 first specifies the part of the document in the captured image acquired by the imaging unit 10, and based on the area information indicating the specified part of the document, the left and right sides of the document area are identified. Extract the tangent slope along the contour. Then, the defect inspection range of the document is determined based on the extracted tangent slope. The details of the operation of the extraction area specifying unit 25 will be described later in detail.

  The perspective conversion unit 26 is caused by taking an image from obliquely above (hereinafter also referred to as a tilt scan) based on the shape of the region indicating the document G of the captured image acquired by the imaging unit 10 (hereinafter also referred to as a document region shape). A distortion correction coefficient for correcting the geometric distortion (geometric distortion) of the captured image is obtained, and the geometric distortion of the captured image is corrected based on the obtained distortion correction coefficient.

  For example, in the perspective conversion unit 26, the rectangular document G is deformed into a trapezoid on the image data captured by the tilt scan. The perspective conversion unit 26 performs processing to return this to a rectangle. For this reason, first, a trapezoidal document area shape is extracted from the captured image by detecting the entire edge (edge) of the document image using an edge detection filter or the like, and a correction coefficient before perspective transformation is calculated from the hypotenuse of the trapezoid. Then, the trapezoidal document area is geometrically converted into a rectangular shape that matches the actual shape.

  Note that, in the captured image processing unit 20, for example, as commonly proposed by the applicant in Japanese Patent Application No. 2001-334163, an edge generally used in image processing on a captured image acquired by the imaging unit 10 is used. Applying an emphasis filter or the like to improve sharpness using a predetermined correction coefficient, a noise reduction process to reduce the noise component of the captured image, or applying a smoothing filter, for example, to remove moire Or a functional unit for performing a correction process such as a noise removal process for performing a smoothing process for smoothing halftone data.

  When performing enhancement processing, smoothing processing, or the like, it is preferable to adjust the parameters of the respective filters with correction coefficients. In this case, it is more preferable to change both the sharpness and the degree of noise removal for each line, that is, it is preferable to perform the enhancement process and the noise removal process simultaneously for each line.

  FIG. 3 is a diagram illustrating a relationship between a captured state and a captured image that is read, and illustrates a captured state of an image that is read by the camera head 12 (A), a captured image (B), and a region It is a figure (C) which shows the image cut out by cutout part 28. Here, a case where the focus is set to the front edge of the document G when shooting is shown. However, the present invention is not limited to this. For example, the focal point may be focused on the back edge or the center of the paper. Further, the description will be made assuming that there is no curl distortion component.

  As shown in FIG. 3A, the camera head 12 captures the original G by tilt scanning in a state where the front edge of the original G (thick line in the figure) is in focus. Therefore, as shown in FIG. 3B, the obtained captured image is a trapezoidal image having an image height ch and image widths cw1 and cw2. The perspective conversion unit 26 geometrically converts a trapezoidal image as shown in FIG. 3B into a rectangular image having an image height dh and an image width dw as shown in FIG. 3C.

  FIG. 4 is a flowchart showing an outline of a processing procedure in the image reading apparatus 3. Steps S380 and S382 will be described in the second embodiment.

  When an imaging start command is issued from the operator to the image reading device 3, a shooting trigger is sent to the CCD image pickup device 12 a of the camera head 12 from a control unit (for example, CPU) that controls each unit of the image reading device 3 (not shown). (S100). When the CCD imaging device 12a receives this shooting trigger, it captures an image of the document G placed on the document table 16 and acquires a captured image (S102). The captured image captured by the camera head 12 is temporarily stored in the data storage unit 30.

  Next, the shading correction unit 22 reads the captured image from the data storage unit 30, and performs shading correction for correcting the illuminance unevenness and the sensor pixel characteristic variation as described above (S104). Next, the optical distortion correction unit 24 performs optical distortion correction on the image subjected to the shading correction as described above to correct distortion aberration of the photographing lens 12b (S106).

  Next, the extraction area specifying unit 25 sets, as an effective extraction area, an area from which the curl distortion component caused by warping or bending of the paper surface is excluded from the image subjected to the optical distortion correction by the optical distortion correction unit 24 (S110). ).

  Next, the perspective conversion unit 26 corrects the geometric distortion of the image caused by the tilt scan with respect to the image in which the optical distortion is corrected and the curl distortion component is removed. At this time, by fixing the number of pixels on the lower base of the trapezoid and extending the upper base, the amount of blurring of the processed image is minimized by the interpolation processing accompanying the geometric transformation. Next, the perspective conversion unit 26 extracts the coordinates of the rectangular document region from the image subjected to geometric conversion using edge detection or the like (S114). The processing target range in the perspective conversion unit 26 may be the entire range of the captured image, the entire range of the document G, or only the effective range specified by the extraction area specifying unit 25.

  Next, based on the coordinates of the rectangular document area extracted by the perspective conversion unit 26, the area cutout unit 28 uses an area excluding the curl distortion component (three-dimensional distortion component) as an effective rectangular document area. Cut out (S118). Then, the extracted image of the effective area is stored in the data storage unit 30.

  The image data stored in the data storage unit 30 is sent to, for example, the print output processing unit 72 shown in FIG. Alternatively, it is sent to the display output processing unit 82 for display output. In any case, only an effective area from which a three-dimensional distortion component resulting from warping or bending of the paper surface is excluded is printed out or displayed. A portion having a large amount of warping or bending is excluded in advance and output.

<Details of the extraction area identification unit>
FIG. 5 is a block diagram illustrating a configuration example of the extraction area specifying unit 25. The take-out area specifying unit 25 specifies a document area specifying unit 252 that specifies a document part in the captured image acquired by the imaging unit 10, and the area information indicating the document part specified by the document area specifying unit 252. A tangential slope extraction unit 254 that extracts the slope of the tangent along the contours of the left and right sides of the region.

  Further, the extraction area specifying unit 25 determines the limit area that defines the area having the curl distortion component caused by warpage or deflection of the paper based on the tangential inclination extracted by the tangential inclination extraction unit 254, thereby determining the distortion area. A curl distortion region specifying unit 256 to be specified, an allowable range setting unit 260 for setting an allowable range for determining a determination condition when specifying a strain region based on a user instruction, and a limit coordinate determining condition in the curl distortion region specifying unit 256 And a limit coordinate determination condition setting unit 262.

  Further, an effective area specifying section 258 for specifying an effective area of the original to be taken out by removing an area having the curl distortion component specified by the curl distortion area specifying section 256 from the original portion specified by the original area specifying section 252, and a user instruction And an effective region specifying condition setting unit 264 for setting an effective region specifying condition in the effective region specifying unit 258.

  FIG. 6 is a flowchart illustrating an example of a processing procedure in the extraction area specifying unit 25, that is, a detailed example of step S110 illustrated in FIG.

  First, the document area specifying unit 252 specifies the portion of the document in the captured image acquired by the imaging unit 10 based on the contour information (S140). Thereafter, the document area specifying unit 252 extracts vertex information from the contour information extracted from the document information (S142). In general, printed sheets discharged from a printing apparatus rarely bend even if they are warped or bent, and thus the apexes of the four corners of the sheet can be specified by such a method.

  For example, the document area specifying unit 252 detects the outline of the area of the document G (printed paper) from the captured image that has been subjected to the optical distortion correction by the optical distortion correction unit 24. For example, since the color of the upper surface of the document table 16 and the document G are different, the region image of the document G (hereinafter referred to as a paper region image) in the captured image can be specified using this color difference.

  For this process, a known method can be used. As an example, an edge detection filter is applied to a captured image that has been corrected for optical distortion, and among the edges obtained thereby, the outermost closed edge is used as the paper outline, and the inside is used as the paper region image. There are ways to do it.

  When the document area specifying unit 252 detects the outline of the document G and the paper area image surrounded by the outline from the captured image, the tangential inclination extracting unit 254 next detects the paper area image specified by the document area specifying unit 252. Based on the above, the inclination of the tangent line is extracted along the contour of the right side of the document area by using the edge tracing method (S144).

  When a three-dimensional distortion has occurred in the document G due to warping or bending, the contour of the document G specified by the document area specifying unit 252 is curved as shown in FIG. Occurs. Based on the contour information, the tangential slope extraction unit 254 obtains the slope of the straight line representing the deviation from the straight line that determines the trapezoid. This process will be described in detail later.

  Next, the curl distortion region specifying unit 256 determines a region where the tangent slope extracted by the tangential slope extraction unit 254 is larger than a predetermined reference value according to the determination condition set by the limit coordinate determination condition setting unit 262. It is determined that the region has a curl distortion component caused by warping or bending (S146).

  Next, the effective area specifying unit 258 has a curl distortion component specified by the curl distortion area specifying unit 256 from the document portion specified by the document area specifying unit 252 according to the specifying condition set by the effective area specifying condition setting unit 264. By excluding the area, only the remaining area is set as an effective area, and the area information is notified to the subsequent perspective conversion unit 26 and the area cutout unit 28 (S148). That is, area coordinates whose inclination is within the threshold are extracted, and a portion determined by the area coordinates is set as an effective area.

  FIG. 7 is a diagram schematically illustrating an example of a captured image obtained by photographing a printed sheet on the document placement table 16. When the printed paper (original G) on the original setting table 16 is photographed obliquely from above with the camera head 12, if the original G is not warped, bent or bent, the rectangular original G is As shown in FIG. 3, the captured image data 401 should be transformed into a trapezoidal shape 402 indicated by a broken line 402 on the captured image data.

  On the other hand, when a three-dimensional distortion is generated in the document G due to warping or bending, the image of the document G in the captured image data has a curved shape on the outline of the paper as indicated by the solid line 403. Distortion occurs and the shape is slightly deviated from the trapezoid.

  FIG. 7 shows an example of a paper area image when the printed paper is only warped or bent and there is no bending, and points A, B, C, and D are the apexes at the four corners of the paper. It corresponds to.

  FIG. 8 is a diagram for explaining tangent extraction processing by the tangent slope extraction unit 254 which is the main processing in the extraction area specifying unit 25. FIG. 8 shows the concept of a method for calculating the tangent slope when the edge CD is taken as an example from the outline 403 of the paper region image shown in FIG. FIG. 8 shows the state of the right side CD when the horizontal axis is the y coordinate and the vertical axis is the x coordinate. In FIG. 2A, a thick line is a contour line LCD of the document connecting the end point C to the end point D.

  Here, the boundary of the document area is obtained as a pixel row by edge detection in the document area specifying process by the document area specifying unit 252. For example, by applying a predetermined edge extraction filter, the operator “i” is initially set to “0”, and the edge coordinate x (i) when the y coordinate is y (i) is detected. This causes (x, y) to scan as (0, y (i)), (1, y (i)), (2, y (i)), (3, y (i)),. Thus, it is preferable that the gradation value is first “255”, that is, the point at which the black color changes first is detected as the edge coordinates (x (i), y (i)).

  Next, the tangential slope extraction unit 254 traces the pixel row from the end point C toward the end point D with respect to all the extracted edge coordinates, for example, FIG. 8 shows points P1 (x1, y1) and P2 (x2, As shown by y2), the slope a (i-1) between the adjacent edge coordinates (x (i-1), y (i-1)) and (x (i), y (i)) is expressed by the equation By performing the calculation according to (3), the inclination a of the contour line is traced.

  The tangential slope extraction unit 254 performs the same processing for the left side AB, the upper side AD, and the lower side BC.

  FIG. 9 is a diagram for explaining a distortion area specifying process in the curl distortion area specifying unit 256. FIG. 9 is a graph in which the slope a of the tangent line of the contour line LCD of the right side CD is plotted, and shows the rate of change of the slope a.

  If no three-dimensional distortion has occurred in the document G due to warping or bending, the contour line LCD of the document is a straight line having no curved distortion, and the inclination a obtained by the tangential inclination extraction unit 254 is It should be constant without location dependency. The inclination when the original is not distorted is known in advance from the geometrical arrangement of the camera head 12 and the original G. This inclination is hereinafter referred to as a reference inclination A.

  On the other hand, when a three-dimensional distortion has occurred in the document G due to warping, bending, etc., the contour LCD of the document has a curved distortion, and the slope required in the distorted portion is: It has location dependence and is smaller or larger than the reference inclination A determined from the geometrical arrangement of the camera head 12 and the original G described above.

  Even if a three-dimensional distortion occurs due to warping or bending of the original G, it is sufficient that there is no inconvenience in character recognition performance, and distortion is allowed to some extent, but if there is distortion above a certain level, No longer allowed. The inclination of the tangent corresponding to the distortion at the allowable limit is obtained in advance, and this is set as a threshold value (allowable range) ΔA as a determination condition. This allowable range ΔA can be set by the user via the allowable range setting unit 260.

  As shown in FIG. 9A, the curl distortion region specifying unit 256 plots the tangent slope a with respect to the y-axis obtained by the tangential slope extraction unit 254, and coordinates (limit of the limit region) within A ± ΔA. Coordinates) Y1CD, Y2CD are stored in the data storage unit 30, for example.

  The curl distortion region specifying unit 256 performs the same processing on the right side AB corresponding to the left side CD, and similarly stores the limit coordinates Y1AB and Y2AB in the data storage unit 30. Further, the curl distortion region specifying unit 256 performs the same processing on the upper side AD and the lower side, and similarly stores the limit coordinates X1AD, X2AD, X1BC, and X2BC in the data storage unit 30.

  In other words, the curl distortion region specifying unit 256 coordinates Y1CD, Y2CD, Y1AB, Y2AB, X1AD, X2AD, X1BC, X2BC (collectively Y1, Y2, X1, X2) that define the allowable limits stored in the data storage unit 30. ) Of the document portion specified by the document region specifying unit 252, the outside of the limit coordinates is set as a region having a curl distortion component caused by warping or bending of the paper surface. In short, the curl distortion region specifying unit 256 specifies a region having a three-dimensional distortion component by comparing the tangential slope a extracted by the tangential slope extraction unit 254 with the threshold value ΔA. Actually, it has a curl distortion component within the allowable limit, but since it is within the allowable range ± ΔA, it is ignored and handled as having no curl distortion component.

  Here, there are various ways of bending the contour line that specifies each side of the document G, but generally, there is little three-dimensional distortion at the center of the document, and the contour line is generally a straight line. You can think about it. On the other hand, three-dimensional distortion is large near both ends of the document. Further, although the way of bending gradually increases from the center side, it may be considered that there are many cases that stably and monotonously increase as shown in FIG. 9B. However, as shown in FIG. 9C, there is a case where there is a wave.

  Therefore, the curl distortion region specifying unit 256 specifies each true limit coordinate based on the slope a extracted for each of the right side, the left side, the upper side, and the lower side specified by the tangential slope extraction unit 254. The true limit coordinates are stored in the data storage unit 30.

  As a method for specifying this true limit coordinate, a slope tangent a to the x-axis (in the case of the upper side or the lower side) or the y-axis (in the case of the right side or the left side) is plotted, and as shown in FIG. Searching in both directions from the center to the outside and adopting the first coordinates Y1, Y2 (or X1, X2) outside A ± ΔA as true limit coordinates, or as shown in FIG. There is a method of searching from the two vertices to the center side and adopting the first coordinates Y1, Y2 (or X1, X2) within A ± ΔA as true limit coordinates.

  Regardless of which method is used, even if the case shows a stable monotonic increase as shown in FIG. 9B or a case with undulation as shown in FIG. When it is smaller than the range ± ΔA, the obtained limit coordinates Y1, Y2, X1, and X2 are the same.

  FIG. 10 is a diagram for explaining the effective area specifying process by the effective area specifying unit 258 in the extraction area specifying unit 25.

  The effective area specifying unit 258 sets only the inside of the original portion specified by the original area specifying unit 252 excluding the outside of the limit coordinates Y1 and Y2 stored in the data storage unit 30 as the effective area of the original.

  Here, in general, the contour lines that specify each side of the document G have different curves. Therefore, the effective region is specified by obtaining coordinates that define the allowable limit for each of the left side AB, right side CD, upper side AD, and lower side BC. Then, as shown in FIG. 10 (A), the effective areas in the y-axis direction specified by the left side AB and the right side CD are different from each other. Similarly, as shown in FIG. 10 (B), the upper side AD and the lower side BC The specified effective areas in the x-axis direction are also different.

  Therefore, the effective area specifying unit 258 specifies each effective area based on each limit coordinate viewed from each of the right side, the left side, the upper side, and the lower side specified by the tangential slope extraction unit 254, and sets each effective area. Based on this, the true effective area is identified.

  As a method for specifying the true effective area, as shown in FIG. 10C, a method in which a common part of each effective area is set as the true effective area, that is, a method of taking an AND (logical product) of logical operations, Alternatively, as shown in FIG. 10D, the right side and the left side and the upper side and the lower side are ORed (logical sum), and their AND (logical product) is a true effective area, that is, OR of logical operations. There is a method of combining (logical sum) and AND (logical product).

  Further, there is a method in which all the effective areas are set as true effective areas, that is, a method of taking OR (logical sum) of logical operations. Furthermore, as a method of combining OR (logical sum) and AND (logical product), for example, the right side and the left side are AND (logical product) while the upper side and the lower side are There is also a method of taking OR (logical sum), or conversely, taking OR (logical sum) for the right side and the left side and taking AND (logical product) for the upper side and the lower side. Of course, the effective area may be set based only on the limit coordinates of one of the left side AB, the right side CD, the upper side AD, and the lower side BC.

  Which method should be adopted may be determined according to the purpose. For example, if accuracy is important, a method of taking AND (logical product) for all sides may be adopted, and a method of taking OR (logical sum) for all sides may be adopted to make the effective area as wide as possible. . Regardless of which identification method is employed, the effective area can be reliably identified.

  As described above, according to the image reading device 3 of the above-described embodiment, the three-dimensional shape such as the warp or the deflection of the document G to be read from the region of the document G in the captured image captured by the imaging unit 10. It is possible to invalidate a region having a large distortion. Only a region with little curl distortion can be perspective-transformed as an effective region, and can be extracted. A distance measuring mechanism is unnecessary, and distance calculation and correction coefficient calculation processing based on the distance calculating mechanism are not necessary.

  Further, by providing the allowable range setting unit 260, the limit coordinate determination condition setting unit 262, and the effective region specifying condition setting unit 264, the user can adjust the effective region to be extracted according to the purpose.

Second Embodiment
FIG. 11 is a diagram showing a second embodiment of the image processing system, and is a schematic diagram when the image processing system 1 shown in FIG. 1 functions as an image inspection apparatus according to the present invention. Here, the installation state of the camera head 12 is mainly shown.

  As shown in FIG. 11, the image inspection apparatus 5 includes an image reading unit 100 corresponding to the imaging unit 10, an image inspection processing unit 300 that inspects whether an image read by the image reading unit 100 is defective, an image reading The data storage unit 400 corresponding to the data storage unit 30 that stores the image read by the unit 100, the reference image used for inspection by the image inspection processing unit 300, or intermediate data during data processing, and the image reading unit 100 A printer unit 500 is provided for outputting the obtained image to a predetermined recording medium.

  The printer unit 500 is operable together with various functions of the image reading unit 100. In order to operate the image processing system 1 as a digital printing system, a print engine similar to the print engine 70 shown in FIG. 570 is arranged.

  The printer unit 500 includes a discharge port 582 and a discharge tray 584. The paper discharge tray 584 is colored so that the document (printed paper) G and the base (color of the paper discharge tray) can be distinguished.

  The image reading unit 100 includes a camera head 102 corresponding to the camera head 12 of the first embodiment that reads an image of the document G. The camera head 102 includes the same functional unit as the camera head 12 of the first embodiment. For example, the camera head 102 includes a photographic lens 104 corresponding to the photographic lens 12 b, a CCD image sensor 106 corresponding to the CCD image sensor 12 a, and a read signal processor 108 corresponding to the read signal processor 18.

  The camera head 102 is supported by a support member (not shown) so that the orientation of the photographing lens 104 with respect to the document G discharged onto the paper discharge tray 584 can be adjusted, and the entire image of the paper discharge tray 584 is captured above the discharge port 582. It is installed to be able to.

  FIG. 12 is a block diagram showing details of the image inspection apparatus 5 shown in FIG. This image inspection apparatus 5 is characterized in that the function of the extraction area specifying unit 25 of the first embodiment is incorporated in the mechanism proposed by the present applicant in Japanese Patent Application No. 2002-252547. This will be specifically described below.

  The image inspection apparatus 5 includes an image reading unit 100 that captures an image and acquires a captured image, an image inspection processing unit 300, and a data storage unit 400.

  The image reading unit 100 includes a captured image processing unit 200 corresponding to the captured image processing unit 20 that performs predetermined processing on the captured image acquired by the camera head 102. The captured image processing unit 200 includes the same functional units as the captured image processing unit 20 of the first embodiment.

  For example, the captured image processing unit 200 includes a shading correction unit 222 that corrects illuminance unevenness and sensor pixel characteristic variation of a captured image read by the camera head 102, an optical distortion correction unit 224 that corrects distortion aberration of the photographing lens, and an inspection. An inspection target region specifying unit 250 that sets a target region, a perspective conversion unit 226 that performs a perspective conversion process, and a region cutout unit 228 that extracts an inspection target region are provided.

  The inspection target area specifying unit 250 corresponds to the extraction area specifying unit 25 of the first embodiment, and includes a processing function unit similar to the extraction area specifying unit 25 inside.

  In the present embodiment, when processing is performed in the image inspection processing unit 300, only an effective region from which a three-dimensional distortion component due to warping or bending of the paper surface is excluded is to be inspected. For example, the coordinate data indicating the effective inspection region specified by the inspection target region specifying unit 250 is stored in the data storage unit 400 without the region cutting unit 228, and the image inspection processing unit 300 side Based on the coordinate data indicating the effective inspection area stored in the data storage unit 400, only the effective area from which the three-dimensional distortion component is excluded may be subjected to the image inspection.

  In addition, as a mechanism for performing error correction for reducing an error between a captured image read by the camera head 102 and a reference image for inspection in the image reading unit 100, for example, the applicant of the present application is Japanese Patent Application No. 2001-334163. A correction coefficient calculation unit and an image correction processing unit as proposed in the above issue may be provided.

  The correction coefficient calculation unit and the image correction processing unit perform enhancement processing (sharpness processing) and noise removal processing for improving sharpness on the captured image acquired by the camera head 102. Note that not only the image acquired by the camera head 102 but also the reference image processing unit 310 (to be described later) converts the resolution in the same manner as the captured image and performs a blurring process (smoothing process). You may make it reduce the error at the time of comparing an image (detailed explanation is omitted).

  The data storage unit 400 holds a captured image acquired by the camera head 102, a processed image processed by the captured image processing unit 200, or various intermediate calculation results used for processing in the image inspection.

  The image inspection processing unit 300 performs a division process on the reference image to be compared in the image inspection, and changes the division size as necessary, and the division output from the reference image processing unit 310. An image inspection unit 330 that inspects the quality of the captured image based on the image and a control unit 350 that controls each unit of the image inspection apparatus 5 are provided.

  The reference image processing unit 310 includes an image dividing unit 312 that divides the reference image into a predetermined size, an index value calculating unit 314 that calculates the distribution or standard deviation of the image data of each divided reference image as an index value, and an index value An index value determination unit 316 that determines whether the variance or standard deviation calculated by the calculation unit 314 is within a specified value is provided.

  In addition, the reference image processing unit 310 changes the size by the divided image size changing unit 320 and the divided image size changing unit 320 that change the divided size by changing the image size of the divided reference image by a certain increment. In response to this, based on the size of the reference image changed by the divided image size changing unit 320, the divided arrangement and size are reconfigured for the reference image excluding the divided reference image after the size change, that is, the reference image A divided image rearrangement size changing unit 322 that updates the division method as a whole is provided.

  In the reference image processing unit 310, a reference image (original image data) serving as a reference for image inspection stored in the data storage unit 400, as proposed by the applicant in Japanese Patent Application No. 2001-334163. Based on the coordinates of the rectangular document area cut out by the area cutout unit 228 of the image reading unit 100, for example, a resolution conversion unit 324 that performs resolution conversion processing using a linear interpolation method or the like, or resolution conversion processing is performed. It is more preferable to provide a blurring processing unit 326 that performs a blurring process for reducing high-frequency components on the captured image.

  The image inspection unit 330 includes an alignment processing unit 332 that aligns the position of the captured image acquired by the camera head 102 and the processed (region-divided) reference image output from the reference image processing unit 310, and an alignment process. A defect abnormality detection processing unit 334 that detects a defect or an image quality abnormality of a captured image acquired by the camera head 102 based on the image aligned by the unit 332.

  FIG. 13 is a flowchart showing an outline example of a processing procedure in the image inspection apparatus 5. FIG. 14 is a diagram for explaining that an area with a large curl distortion is excluded from a defect detection target area. FIG. 14 illustrates the case where the right side of the captured image is the determination target.

  In the image inspection apparatus 5 according to the present embodiment, when the printer unit 500 is in a standby state, the camera head 102 periodically takes an image on the paper discharge tray 584 according to an instruction from the control unit 350 and outputs the image.

  The control unit 350 compares the image on the paper discharge tray 584 stored in the data storage unit 400, which is an example of a reference image, with the captured image acquired by the camera head 102, and the document is on the paper discharge tray 584. If the document is on the paper discharge tray 584, the image inspection flag is turned off and the standby state is continued. If there is no document on the paper discharge tray 584, the image can be inspected. The flag is turned on and the standby state is continued (S300, S302-NO).

  When the control unit 350 receives a print job in the standby state (S302—YES), the control unit 350 determines a flag that allows image inspection (S304). If the image inspection flag is off (S304-off), the control unit 350 causes the print engine 570 to perform normal printing processing (S306), and returns to the standby state (S308). On the other hand, when the flag indicating that image inspection is possible is on (S304—on), the control unit 350 performs image inspection on the Nth sheet (first of course, the first sheet) as follows.

  That is, the control unit 350 first causes the print engine 570 to start print processing of the Nth sheet (naturally the first sheet at first) (S310), and the developed image data to the print output processing unit 572 of the print engine 570. The data is sent out and stored in the data storage unit 400 (S312). When the rendering processing of the last image data is completed, the first sheet processing completion signal from the print output processing unit 572 is waited (S314). When the completion signal is received, a shooting start trigger signal is sent to the camera head 102 after a specified time. (S314-YES, S316).

  The camera head 102 releases the shutter in synchronization with the trigger signal, but immediately after the document is output on the paper discharge tray 584 at this time. The captured image processing unit 200 performs preprocessing on the captured image captured by the camera head 102 (S320).

  The preprocessing for the captured image is the same as the processing (steps S104 to S114) of the captured image processing unit 20 in the image reading device 3 of the first embodiment. For example, the shading correction unit 222 reads a captured image from the data storage unit 400 and performs shading correction for correcting unevenness in illuminance and sensor pixel characteristic variation (S322).

  Next, the optical distortion correction unit 224 performs optical distortion correction for correcting distortion aberration and the like of the photographing lens 104 on the image subjected to the shading correction (S324).

  Next, the inspection target area specifying unit 250 sets, as an effective extraction area, an area from which the curl distortion component due to warping or bending of the paper surface is excluded from the image subjected to the optical distortion correction by the optical distortion correction unit 224 ( S326).

  Next, the perspective conversion unit 226 corrects the geometric distortion of the image caused by the tilt scan with respect to the image in which the optical distortion is corrected and the curl distortion component is removed. At this time, it is preferable to minimize the amount of blurring of the processed image by the interpolation processing accompanying the geometric transformation by fixing the number of pixels on the lower base of the trapezoid and extending the upper base.

  Next, the perspective conversion unit 226 uses the difference between the color of the background paper discharge tray 584 and the white of the edge of the document G to perform perspective conversion using edge detection that extracts the coordinates of the rectangular document region. The coordinates of the rectangular document area are extracted from (S328). Next, the area cutout unit 228 cuts out the rectangular document area based on the coordinates of the rectangular document area extracted by the perspective conversion unit 226 (S329). The area cutout unit 228 temporarily stores the cut out image in the data storage unit 400.

  In the case where there are a plurality of print jobs (N sheets), following the preprocessing (to S320), the control unit 350 performs the remaining print processing in parallel with the above-described processing until all the plurality of sheets (N sheets) are completed. (S330-NO).

  When all the plural sheets (N sheets) are completed, the control unit 350 returns to the standby state (S330-YES, S308). In this way, all captured images captured by the camera head 102 and preprocessed by the captured image processing unit 200 are converted into digital signals and stored in the data storage unit 400.

  On the other hand, in the reference image processing unit 310, in parallel with the imaging by the camera head 102, first, when the reference image processing unit 310 is configured to include the resolution conversion unit 324 and the blur processing unit 326, resolution conversion is performed. The unit 324 uses, for example, a linear interpolation method based on the coordinate data indicating the rectangular document area extracted by the perspective conversion unit 226 with respect to the reference image for image inspection stored in the data storage unit 400. Then, resolution conversion processing is performed (S334).

  Subsequently, the blurring processing unit 326 performs a blurring process for reducing high frequency components on the image subjected to the resolution conversion processing by the resolution conversion unit 324, and stores the processed image in the data storage unit 400. (S336).

  The reason why the reference image (original image data) is subjected to the blurring process is that the reference image has an ideal gradation characteristic, and therefore cannot be removed from the captured image captured by the camera head 102. This is to minimize the error in the comparison process (difference process) between the reference image and the read image performed in the image inspection process.

  Next, the image inspection processing unit 300 performs a reference image on which image processing (resolution conversion and blurring processing) has been performed, and a captured image in which image correction processing (enhancement processing & noise removal processing) has been performed on the captured image. Are read from the data storage unit 400, and a difference process for comparing the two is performed to determine an image defect or the like.

  Therefore, first, in the reference image processing unit 310, the image dividing unit 312 divides the reference image for image inspection stored in the data storage unit 400 into a specified size (S340). The division of the reference image performed here gives an initial value of the division size. For example, the division is appropriately determined such as 10 divisions in the image width direction (main scanning walking) and 10 divisions in the image height direction (sub-scanning direction). be able to.

  Next, the index value calculation unit 314 calculates the variance or standard deviation of the image data for each of the divided reference images (divided images) (S342). Then, based on Expression (4), a normalized cross-correlation C used for alignment and image defect detection is obtained as an index value.

  The first term of the denominator in Equation (4) is proportional to the variance of the captured image, and the second term is proportional to the variance of the reference image. In order for the normalized cross-correlation C not to be indefinite and to keep the calculation accuracy above a certain level, whether the second term is not “0” and is a value that does not decrease the calculation accuracy (hereinafter referred to as the prescribed value TH1). Can be determined. Therefore, the index value determination unit 316 determines the variance or standard deviation calculated by the index value calculation unit 314 by the index value determination unit 316 based on this criterion (S322).

  If the variance or the standard deviation is less than the prescribed value TH1 (S344-NO), the divided reference image contains no image data such as characters or only a part of it, so the divided image size is changed. The unit 320 enlarges the division size by a predetermined increment (in at least one of the main scanning direction and the sub-scanning direction) (S346).

  Then, the index value calculation unit 314 calculates again the variance or standard deviation for the divided image after the size change (S342), and the index value determination unit 316 determines the normalized cross-correlation C and the threshold value TH1 (S344). ). This is repeated until the normalized cross-correlation C does not become indefinite and the calculation accuracy of a certain level or more is maintained, that is, until the normalized cross-correlation C exceeds the specified value TH1 (NO in S344).

  If a portion such as a character is included in the divided image, the variance or the standard deviation exceeds the specified value. Therefore, the divided image rearrangement size changing unit 322 determines the divided size at that time (S348), and the divided portion having the determined size. Are subdivided (S352). At this time, an overlap with the already determined region is prevented from occurring. Each unit of the reference image processing unit 310 repeats the above-described processing for the undecided divided image (S342 to S350-NO).

  In this way, by changing the division size of the reference image so that the normalized cross-correlation coefficient C does not become indefinite (that is, the calculation error becomes small), the evaluation value of the divided image (dispersion or standard in the previous example). Since the deviation) is surely greater than or equal to the specified value, the accuracy of alignment (S360) between a reference image and a captured image, which will be described later, is improved.

  When the above processing is completed over the entire reference image (S350—YES), the alignment processing unit 332 divides the reference image processing unit 310 and resizes the reference image as necessary, and the image reading unit 100. The process for aligning the captured image acquired in step S360 is started (S360).

  For example, the alignment processing unit 332 sequentially calculates the normalized cross-correlation coefficient C while scanning the captured image using the divided reference image as a template (S362), and the location with the highest correlation coefficient is aligned. Positioning is performed by so-called template matching, which is regarded as a part.

  Next, the defect abnormality detection processing unit 334 performs defect abnormality detection processing for detecting an image defect or image quality abnormality. For example, when the normalized cross-correlation coefficient C at the position where the position is correct falls below the determination index value TH2, it is assumed that there is an image defect in that portion, and defect abnormality detection processing is performed (S364-YES, S366).

  For example, if there is a defect pattern such as a black spot in a print output, that is, a captured image, the normalized cross-correlation coefficient C clearly decreases at a place including the defect, so that it can be easily determined that a defect exists. The alignment processing unit 332 and the defect abnormality detection processing unit 334 perform the above processing for all the divided reference images (S368-NO), and after completion, return to the standby state (S368-YES, S308).

  When the defect abnormality detection processing unit 334 detects an image defect, the image inspection apparatus 5 issues a warning on a user interface (not shown) such as a predetermined display device or an audio signal generator (for example, a speaker) or print management. Using software or the like, a warning is issued to a client terminal (for example, a personal computer) that is going to print through the communication network, and further, a printer maintenance company is notified through the communication network via a remote maintenance system or the like.

  Here, when image defect detection is performed, distortion is allowed to some extent depending on the division size of the reference image. However, if there is distortion above a certain level, defect word detection occurs. Therefore, as a process unique to the present embodiment, the inspection target area specifying unit 250 preliminarily calculates the inclination of the tangent corresponding to the distortion at the allowable limit in the same manner as the process in the captured image processing unit 20 of the first embodiment. This is obtained and this is set as ΔA.

  Then, the slope of the tangent line with respect to the y-axis is plotted, and the coordinates (limit coordinates) X1, X2, Y1, Y2 of the region within A ± ΔA are stored. Thereafter, the perspective transformation process is performed on the captured image by the perspective transformation unit 226, and at the same time, the coordinates of X1, X2, Y1, and Y2 are transformed according to the perspective transformation formula to be X1 ′, X2 ′, Y1 ′, and Y2 ′. . Next, when an image defect is diagnosed by performing a difference process in the image inspection processing unit 300, for example, the difference process is performed only in a range where the y coordinate is Y1 'to Y2'.

  That is, as described with reference to FIG. 4 of the first embodiment, based on the paper region image specified by the document region specifying unit 252, the tangent slope extracting unit 254 determines the tangent along the outline of each side of the document region. The inclination is extracted (S144), and the curl distortion region specifying unit 256 sets a region where the tangent gradient extracted by the tangent gradient extraction unit 254 is larger than a predetermined reference value to a relatively large curl distortion caused by warpage or deflection of the paper. It is determined that the region has components (S146).

  Thereafter, the effective area specifying unit 258 excludes the area having the curl distortion component specified by the curl distortion area specifying unit 256 from the document portion specified by the document area specifying unit 252 and sets only the remaining areas as inspection targets. The effective region, that is, the coordinates of the region within the threshold with respect to the tangent slope data calculated by the tangential slope extraction unit 254 is extracted, and the information is stored in the data storage unit 400, so that The output unit 228 is notified (S148).

  In response to this, as shown in FIG. 6, the perspective conversion unit 226 performs a perspective conversion process on the document area, and generates a captured image converted into a rectangular shape having the same size as the original image (S112).

  Based on the region coordinates stored in the data storage unit 400, the image inspection unit 330 captures the original image stored in the data storage unit 400 and the captured image after processing only for the region where the tangent slope a is within the threshold ΔA. Difference processing with the image is performed (S380). Thereafter, based on this difference result, an image inspection is performed to detect the presence or absence of defects in the image of the original G taken by the camera head 102 of the image reading unit 100 (S382).

  As a result, as shown in FIG. 14, it is possible to detect a defect only in a defect detection target region, and it is possible to automatically exclude a region having a large curl distortion. That is, it is possible to realize a highly accurate image defect inspection mechanism that is not affected by an area where the curl distortion is large. A distance measuring mechanism is unnecessary, and distance calculation and correction coefficient calculation processing based on the distance calculating mechanism are not necessary.

  As described in the first embodiment, the processing unique to this embodiment for excluding such a region having a curl distortion component from the inspection target may be the determination target only on the left side, and further, A combination of coordinate values on the right side may be used.

  As described above, according to the configuration of the second embodiment, the normalization cross-correlation calculation is not indefinite or the calculation error is reduced when detecting image defects by registration and normalized cross-correlation. Since the division size of the image is changed so that the evaluation value of the divided image becomes equal to or larger than the specified value, the problem that the normalized cross-correlation calculation becomes indefinite or the calculation error becomes large can be solved. As a result, the alignment accuracy between the reference image and the captured image is improved, and an image inspection apparatus with good image defect detection accuracy can be realized.

  In addition, when the scanned image is inspected by comparing the reference image with the scanned image, the document G to be scanned is read from the area of the document G in the captured image captured by the camera head 102 of the image reading unit 100. A region having a large three-dimensional distortion such as warping or bending can be excluded from the inspection target. Only an area with less curl distortion can be perspective-transformed as an effective area, and it can be taken out and only an area with less curl distortion can be set as an inspection target.

  In other words, with a simple apparatus and processing method, an inspection target that can ensure the accuracy of image defect detection by detecting a boundary where the defect detection accuracy decreases due to three-dimensional distortion such as warping or bending of the document. Since the area can be automatically determined, a highly accurate image defect inspection apparatus can be provided.

  In particular, the mechanism of Japanese Patent Application No. 2002-252547 can be obtained by combining the function of the extraction area specifying unit 25 of the first embodiment with the mechanism proposed by the present applicant in Japanese Patent Application No. 2002-252547. Can compensate for the effects of.

  That is, according to the mechanism of Japanese Patent Application No. 2002-252547, when image defects are detected, the division size of the reference image is adjusted so that the distortion component does not affect the defect detection performance. Can be excluded from the target. However, if there is a distortion above a certain level, there is a possibility that a part with a lot of distortion may be erroneously detected as a defect.

  On the other hand, as in the present embodiment, the inspection target region specifying unit 250 is provided, and the specified non-inspection region having a large three-dimensional distortion is excluded from the target of the image inspection processing, thereby applying Japanese Patent Application No. 2002-252547. Even a portion having a certain distortion or more that causes a problem in the mechanism of the number can be excluded from the inspection target so that erroneous detection does not occur, and erroneous detection of defects can be prevented. That is, accurate defect detection can be performed.

  Note that the applicant of the present application is a Japanese Patent Application 2003-2003, which is capable of creating an image when a sheet to be read is warped in both the vertical and horizontal directions or bent and the image when the sheet is stretched flat is more accurate than before. This is proposed in 280156.

  In the mechanism proposed in Japanese Patent Application No. 2003-280156, when three-dimensional irregularities such as curl occur in the output paper, the nodes are also set inside the paper area image based on the nodes set in the outline of the paper area image. By dividing the paper area in the captured image into meshes using these nodes, and performing perspective transformation processing for each mesh to generate an image viewed from the front, more accurate conversion can be performed. ing. However, in this case, setting of nodes and perspective transformation processing for each mesh are necessary, and the processing becomes complicated.

  On the other hand, in the configuration and method of the present embodiment, when a three-dimensional distortion occurs, a method of reducing the distortion is not used, but an area with a large three-dimensional distortion is excluded from the inspection target. The method of excluding is employed, the processing is relatively simple, and the circuit configuration has the advantage of being simpler than the configuration for reducing distortion.

<Configuration using an electronic computer>
The mechanism of the above-described captured image processing unit 20 in the image reading device 3 and the captured image processing unit 200 in the image inspection device 5, particularly the extraction region specifying unit 25 and the inspection target region specifying unit 250 for performing the distortion region exclusion process are described in ASIC (Application It is not limited to the combination of dedicated hardware processing circuits that perform each function, such as Specified Integrated Circuit), but it is realized by software using a computer (computer) based on the program code that realizes that function. It is also possible to do. By adopting a mechanism for executing processing by software, it is possible to enjoy the advantage that the processing procedure and the like can be easily changed without changing hardware.

  Therefore, a program suitable for realizing the image reading apparatus and the image inspection apparatus according to the present invention to which the processing method described in the above embodiment is applied by software using an electronic computer (computer) or a computer storing this program A readable storage medium can also be extracted as an invention.

  When a series of image processing is executed by software on an electronic computer, a program (such as a built-in microcomputer) in which a program constituting the software is incorporated in dedicated hardware, a CPU (Central Processing Unit), Various functions can be executed by installing a system on a chip (SOC) that implements a desired system by mounting functions such as logic circuits and storage devices on a single chip, or by installing various programs. It is installed from a recording medium in a general-purpose personal computer capable of doing so.

  The recording medium causes a change state of energy such as magnetism, light, electricity, etc. to the reading device provided in the hardware resource of the computer according to the description content of the program, and in the form of a signal corresponding thereto. The program description can be transmitted to the reader.

  The recording medium is distributed to provide a program to the user separately from the computer, and includes a magnetic disk (including a flexible disk FD) on which the program is recorded, an optical disk (CD-ROM (Compact Disc-Read Only Memory). ), DVD (Digital Versatile Disc), including PD), magneto-optical disk MO (including MD (Mini Disc)), or a non-volatile semiconductor memory such as a flash memory or a package medium such as an IC card or a miniature card In addition to being configured by (portable storage medium), it may be configured by a ROM, a hard disk device, or the like in which a program is provided that is provided to the user in a state of being pre-installed in a computer. The program constituting the software is not limited to being provided via a recording medium, and may be provided via a wired or wireless communication network.

  For example, a storage medium that records a program code of software that realizes a function for image processing is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program code stored in the storage medium. By executing the reading, the same effect as that achieved by hardware can be achieved. In this case, the program code itself read from the storage medium realizes an image processing function.

  Further, not only the function of performing image processing is realized by executing the program code read by the computer, but also an OS (operating system; basic software) running on the computer based on an instruction of the program code May perform a part or all of the actual processing and realize a function of performing image processing by the processing.

  Further, after the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. There may be a case where the CPU or the like provided in the card or the function expansion unit performs part or all of the actual processing, and the function of performing image processing is realized by the processing.

  At this time, the program is provided as a file describing a program code that realizes a function of performing image processing. In this case, the program is not limited to being provided as a batch program file, and the hardware of the system configured by a computer Depending on the configuration, it may be provided as an individual program module. For example, it may be provided as add-in software incorporated in existing apparatus control software.

  FIG. 15 configures an image reading apparatus and an image inspection apparatus in software using a CPU and a memory. That is, an image reading apparatus and an image inspection apparatus are realized in software using the functions of an electronic computer such as a personal computer. It is a block diagram which shows an example of the hardware constitutions in the case of doing.

  A computer system 900 constituting a failure diagnosis device or failure diagnosis system includes a controller unit 901, a predetermined external device such as a hard disk device, a flexible disk (FD) drive, a CD-ROM (Compact Disk ROM) drive, or a semiconductor memory controller. And a recording / reading control unit 902 as a driving device for reading and recording data from the storage medium 905. For example, the recording / reading control unit 902 functions as a hard disk device, an FD drive, a CD-ROM drive, or the like.

  The controller unit 901 includes a CPU (Central Processing Unit) 912, a ROM (Read Only Memory) 913 that is a read-only storage unit, and a RAM (Random Access) that is an example of a volatile storage unit that can be written and read at any time. Memory) 915 and RAM (described as NVRAM) 916 which is an example of a nonvolatile storage unit.

  The computer system 900 also includes an instruction input unit 903 having a keyboard, a mouse, and the like as a function unit that forms a user interface, and a display output unit 904 that presents a user with predetermined information such as a guidance screen and a processing result during operation. The image forming unit 906 outputs processed data to a predetermined output medium (for example, printing paper), and an interface unit 909 that functions as an interface between the functional units.

  As the interface unit 909, in addition to the system bus 991 that is a transfer path of processing target data and control data, for example, a camera IF unit 930 that functions as an interface (IIT controller) with the camera head 12 (102), an image A printer IF unit 996 that functions as an interface (IOT controller) with the forming unit 906 and other printers, and a communication IF unit 999 that mediates communication data exchange with the network 9 such as the Internet.

  The communication IF unit 999 is a communication interface with an external device such as a host device. Specific examples of the communication IF unit 999 include a parallel port connected to a printer port of a personal computer and other network interfaces.

  The display output unit 904 includes, for example, a display control unit 942 and a display unit 944 made up of a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display). For example, the display control unit 942 displays guidance information, a failure diagnosis result, and the like on the display unit 944. Note that by using the display unit 944 having the touch panel 932 on the display surface, the instruction input unit 903 for inputting predetermined information with a fingertip or a pen can be configured.

  The image forming unit 906 forms (prints) a visible image on plain paper or thermal paper using an electrophotographic, thermal, thermal transfer, inkjet, or similar conventional image forming process. For example, a raster output scan-based print engine 964 for receiving print output data of yellow Y, magenta M, cyan C, and black K and operating as a digital printing system is provided. The print engine 964 corresponds to the print engine 70 of FIG. 1 or the print engine 570 of FIG.

  The printer IF unit 996 sorts the image data in a format that can be handled by the print engine 964, supplies the image data to the print engine 964, and controls the print engine 964 to execute printing.

  It should be noted that not all processing of each functional part (for example, the tangential slope extraction unit 254 and the curl distortion region specifying unit 256) of the captured image processing units 20 and 200 shown in the above embodiment is performed by software, but these functional units. A processing circuit 908 that performs a part of the above with dedicated hardware may be provided.

  Although the mechanism for performing predetermined processing by software can flexibly cope with parallel processing and continuous processing, the processing time becomes longer as the processing becomes complicated, so that a reduction in processing speed becomes a problem. On the other hand, it is possible to construct an accelerator system with a higher speed by using a hardware processing circuit. Even if the processing is complicated, the accelerator system can prevent a reduction in processing speed and can obtain a high throughput.

  The computer system 900 having such a configuration can be the same as the basic configuration and operation of the image reading device 3 and the image inspection device 5 described in the above embodiment. For example, in such a configuration, the CPU 912 controls the entire system via the system bus 991. The ROM 913 stores a control program for the CPU 912 and the like. The RAM 915 is configured by SRAM (Static Random Access Memory) or the like, and stores program control variables, data for various processes, and the like.

  A program that causes a computer to execute image processing in the image reading device 3 and the image inspection device 5 may be incorporated in the ROM 913 in advance when the device is configured, for example, or a non-volatile semiconductor memory such as a CD-ROM, MO, or flash memory It may be distributed through a recording medium such as a card. Furthermore, the program may be downloaded and acquired from another server or the like via the network 9 such as the Internet, or may be updated. The recording medium can store a program that realizes part or all of the functions of image processing.

  For example, an FD 905a or a CD-ROM 905b as an example of a recording medium can store a part or all of the functions in the captured image processing units 20 and 200 described in the above embodiment. That is, the software installed in the RAM 915 or the like includes functional units such as a tangential slope extraction unit and a curl distortion region specifying unit as software, like the captured image processing units 20 and 200 shown in the above embodiment. Such software may be stored in a portable storage medium such as a CD-ROM or FD as application software for image inspection, or distributed via a network.

  When the captured image processing units 20 and 200 are configured by a computer, the recording / reading control unit 902 reads data or a program from the CD-ROM 905b or the like and passes it to the CPU 912. The software is installed in the hard disk device from the CD-ROM 905b or the like. The hard disk device stores data or a program read by the FD drive or CD-ROM drive forming the recording / reading control unit 902, data created by the CPU 912 executing the program, The program is read and passed to the CPU 912.

  The software stored in the hard disk device is read by the RAM 915 and executed by the CPU 912. For example, the CPU 912 executes the above processing based on programs stored in the ROM 913 and the RAM 915 which are examples of the recording medium.

  For example, first, a document to be image-inspected is read by the camera head of the imaging unit, subjected to signal processing for inspection, and then stored in a nonvolatile storage device such as NVRAM 916 or a hard disk device.

  Next, the reference image previously stored in the data storage unit is read out, subjected to signal processing for inspection, and read by comparing with the read image of the original G once stored in the hard disk device or the like Diagnose whether the image is defective.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. Various changes or improvements can be added to the above-described embodiment without departing from the gist of the invention, and embodiments to which such changes or improvements are added are also included in the technical scope of the present invention.

  Further, the above embodiments do not limit the invention according to the claims (claims), and all combinations of features described in the embodiments are not necessarily essential to the solution means of the invention. Absent. The embodiments described above include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. Even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, as long as an effect is obtained, a configuration from which these some constituent requirements are deleted can be extracted as an invention.

  For example, in each of the above embodiments, the extraction region setting and the inspection target region setting are performed based on the image subjected to the optical distortion correction, and then the perspective transformation is performed, but the extraction region setting and the inspection target region setting are performed. The position where the processing to be performed is arranged is not limited to that shown in the above embodiment. For example, an extraction region specifying unit and an inspection target region specifying unit may be provided after the perspective conversion unit.

  In the second embodiment, the example combined with the mechanism proposed in Japanese Patent Application No. 2002-252547 by the applicant of the present application has been described. However, the inspection target area specifying unit 250 is applied to other configurations. May be.

  In the second embodiment, the case where the image inspection unit 330 of the image inspection processing unit 300 detects a local image defect such as a black spot is described. However, the present invention is not limited to this. It is also possible to detect image quality defects such as.

  In the above embodiment, the tangent slope is extracted by using the edge tracing method. However, the present invention is not limited to this, and the outline indicating the document in the captured image obtained by photographing the document from diagonally above is used. Any method can be used as long as the effective region can be identified by excluding the region having the three-dimensional distortion component of the captured image in the captured image from the processing target. Can be taken.

  For example, a curve equation indicating the contour line (side) of the original G is obtained by using, for example, a high-order (for example, cubic) spline function, etc., and the curve equation is analyzed to extract the slope of the tangent line. It is also possible to specify the rate of change of the tangent slope of the contour line as shown in FIG. In this case, it is preferable to specify a function indicating a curve after performing a smoothing process or the like so as not to be affected by noise and minute swell.

1 is a block diagram showing a first embodiment of an image processing system including an image reading device equipped with an image processing device according to the present invention. It is a block diagram which shows the detail of one Embodiment of an image reading apparatus. It is a figure which shows the relationship between an imaging | photography state and the read captured image. 5 is a flowchart showing an outline of a processing procedure in the image reading apparatus. It is a block diagram which shows the structural example of the extraction area | region specific | specification part. It is a flowchart which shows an example of the process sequence in the extraction area | region specific part. It is a figure which shows typically an example of the picked-up image obtained by image | photographing the printed paper on a document mounting base. It is a figure explaining the tangent extraction process by a tangent inclination extraction part. It is a figure explaining the distortion area | region identification process in a curl distortion area | region identification part. It is a figure explaining the effective area specific process by an effective area specific part. It is a figure which shows 2nd Embodiment of an image processing system. It is a block diagram which shows the detail of the image inspection apparatus shown in FIG. It is the flowchart which showed the example of the outline | summary of the process sequence in an image inspection apparatus. It is a figure explaining excluding the area | region where curl distortion is large from a defect detection object area | region. FIG. 3 is a diagram illustrating an example of a hardware configuration when an image reading apparatus and an image inspection apparatus are configured by software using a CPU and a memory.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Image processing system, 2 ... Image processing part, 3 ... Image reading apparatus, 5 ... Image inspection apparatus, 7 ... Image output terminal, 9 ... Network, 10 ... Imaging part, 12, 102 ... Camera head, 12a, 106 ... CCD imaging device, 12b, 104 ... photographic lens, 14 ... support member, 16 ... document placement table, 18, 108 ... read signal processing unit, 20, 200 ... captured image processing unit, 22, 222 ... shading correction unit, 24, 224 ... Optical distortion correction unit, 25 ... Extraction area specifying part, 26, 226 ... Perspective conversion part, 28, 228 ... Area extraction part, 30, 400 ... Data storage part, 70, 570 ... Print engine, 72, 572 ... Print output processing unit, 100... Image reading unit, 250... Inspection target region specifying unit, 252... Original region specifying unit, 254. ... Effective area specifying part, 260 ... Permissible range setting part, 262 ... Limit coordinate determining condition setting part, 264 ... Effective area specifying condition setting part, 300 ... Image inspection processing part, 310 ... Reference image processing part, 330 ... Image inspection part 332: Positioning processing unit, 334 ... Defect abnormality detection processing unit, 350 ... Control unit, 500 ... Printer unit, 582 ... Ejection port, 584 ... Ejection tray

Claims (11)

An image processing apparatus that performs predetermined image processing on an optically read original image,
Effective by excluding a region having a three-dimensional distortion component of the document in the captured image from the processing target by analyzing a contour of the captured image obtained by photographing the document from obliquely above. An image processing apparatus comprising an effective area specifying unit for specifying an area. The effective area specifying unit includes:
A tangential slope extraction unit that extracts a tangential slope along a curve indicating the outline of the document;
2. A distortion region specifying unit that specifies a region having the three-dimensional distortion component by comparing the inclination of the tangent extracted by the tangent gradient extraction unit with a predetermined threshold value. An image processing apparatus according to 1. The image processing apparatus according to claim 1, further comprising a threshold setting unit configured to set the predetermined threshold. Determination for setting a condition for determining a limit point at which a tangent slope extracted by the tangential slope extraction unit is equal to a predetermined threshold when the strain region specifying unit specifies a region having the three-dimensional distortion component The image processing apparatus according to claim 1, further comprising a condition setting unit. The image according to any one of claims 1 to 4, further comprising an effective region specifying condition setting unit that sets a specific condition when the strain region specifying unit specifies the effective region. Processing equipment. An image reading apparatus for optically reading a document,
An image capturing unit that captures a captured image by photographing the original document from obliquely above;
By analyzing a contour line indicating the original of the captured image acquired by the imaging unit, an extraction area in which an area having a three-dimensional distortion component of the original in the captured image is excluded from an output target is specified. An image reading apparatus comprising: an extraction area specifying unit. The extraction area specifying unit
A tangential slope extraction unit that extracts a tangential slope along a curve indicating the outline of the document;
7. A distortion region specifying unit that specifies a region having the three-dimensional distortion component by comparing the inclination of the tangent extracted by the tangent gradient extraction unit with a predetermined threshold value. The image reading apparatus described in 1. An image inspection apparatus that inspects a defect of the captured image by comparing a captured image obtained by optically reading a document with a reference image for inspection corresponding to the document,
An image capturing unit that captures a captured image by photographing the original document from obliquely above;
By analyzing the outline of the captured image obtained by the imaging unit indicating the original, an effective inspection area is determined by excluding an area having a three-dimensional distortion component of the original in the captured image from the inspection target. An inspection area specifying unit to be
Of the captured images acquired by the imaging unit, the captured image acquired by the imaging unit by comparing the reference image with the image of the effective inspection region specified by the inspection target region specifying unit. An image inspection apparatus comprising: an image inspection unit for inspecting. The inspection target area specifying unit
A tangential slope extraction unit that extracts a tangential slope along a curve indicating the outline of the document;
9. A distortion region specifying unit for specifying a region having the three-dimensional distortion component by comparing the inclination of the tangent extracted by the tangent gradient extraction unit with a predetermined threshold value. The image inspection apparatus according to 1. A program for performing predetermined image processing on an optically read original image,
Computer
Effective by excluding a region having a three-dimensional distortion component of the document in the captured image from the processing target by analyzing a contour of the captured image obtained by photographing the document from obliquely above. A program that functions as an effective area specifying unit that specifies an area. Of the captured images, the image of the effective area specified by the effective area specifying unit and an inspection reference image corresponding to the original are compared to function as an image inspection unit that inspects the captured image. The program according to claim 10.

Images