JP3754401B2 - Image distortion correction method, image distortion correction apparatus and image distortion correction program based on representative point measurement - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an inspection system and an identification system based on an image, in particular, a defect detection system based on a comparison between a non-defective image and an inspection image, and an object identification system based on a comparison between a reference image and a comparative image. The present invention relates to an image distortion correction method when a distortion pattern exists between a reference image and a comparison image.
[0002]
[Prior art]
As a conventional image distortion correction means, there is known a method of correcting an optical system distortion mainly due to a lens aberration or a perspective projection effect by expressing it with a mathematical expression. This method is effective for reproducible targets that can be expressed in mathematical formulas, but it is not possible to express mathematical formulas due to irregularities due to irregularities due to differences in individual subjects. It is not possible to adapt to a simple subject.
[0003]
For image distortion that does not have reproducibility or regularity, a method of dividing an image into a grid and determining a two-dimensional deviation amount in each rectangle is used. However, since the locations suitable for positional deviation measurement in the image are limited, there are many cases where deviation measurement points cannot be set by rectangular division of the image.
[0004]
When the rectangular division of the image does not work, there is a method in which measurement representative points are irregularly arranged at positions suitable for positional deviation measurement, and correction is performed for each triangle composed of the representative points in the vicinity of each representative point. However, it is difficult to set the representative point itself, and there is still a limitation that the portions suitable for the positional deviation measurement are limited as described above. Furthermore, since the distortion pattern differs for each correction target image, it is not possible to prepare a fixed correction formula or correction table in advance, and the processing load is generally large.
[0005]
[Problems to be solved by the invention]
The conventional image distortion correction means using mathematical expression cannot be applied to an object having individual differences or an object incapable of mathematical expression.
[0006]
For example, in an electronic circuit board configured on a flexible base material, irregular distortion occurs due to expansion and contraction of the base material being processed. Further, when shooting an object with a line scan camera, if the speed of the transport system is uneven, the image appears as an expansion / contraction of the image even if the shooting object is stable, causing image distortion.
[0007]
Regarding these irregular image distortions, a positional deviation is measured at the representative point and corrected for each neighborhood of the representative point, or a distortion pattern is determined by interpolating the deviation amount at the representative point. However, there is a problem that there are limited places suitable for two-dimensional positional deviation measurement, and setting of representative points is a difficult task. There is also a problem that the amount of calculation for correction processing increases because of different distortion patterns each time.
[0008]
Therefore, the present invention can automatically arrange representative points at positions suitable for misregistration measurement, and can generate a distortion pattern that smoothly changes from the misregistration amount at the representative points in the entire image in a short processing time. An object of the present invention is to provide an image distortion correction method.
[0009]
As for representative points, it is desirable to be able to measure a two-dimensional displacement at each representative point, but in reality, there are often few places suitable for two-dimensional measurement in an image. Accordingly, the present invention realizes representative point automatic placement that can place one-dimensional positional deviation measurement points in addition to two-dimensional positional deviation measurement points, and the measurement results of the two-dimensional positional deviation measurement points and the one-dimensional positional deviation measurement points. An object of the present invention is to provide an image distortion correction method capable of determining a distortion pattern of the entire image by combining the above.
[0010]
The conventional method for determining the distortion pattern of the entire image from the amount of misalignment at the representative point is that a defect occurs at some measurement representative points in the inspection image or comparison image, or some measurement representative points are blocked by foreign matter. If the displacement measurement fails, the distortion pattern cannot be determined. Therefore, the present invention detects a failure even if the position deviation measurement fails at some of the deviation measurement representative points, and compensates the deviation amount of the measurement representative point at which the position deviation measurement has failed from the surrounding measurement points. An object of the present invention is to provide a method for correcting image distortion.
[0011]
It is another object of the present invention to provide an image distortion correction apparatus that realizes the image distortion correction method.
[0012]
Another object of the present invention is to provide an image distortion correction program for causing a computer to execute the image distortion correction method.
[0013]
[Means for Solving the Problems]
In order to achieve the above object of the present invention, according to the first aspect of the present invention, the distortion pattern of the comparison image with respect to the reference image is interpolated from the deviation amount measured by pattern matching at the corresponding measurement representative point. An image distortion correction method for calculating is provided. This image distortion correction methodSet the measurement representative points corresponding to the reference image and the comparison image, andAn auxiliary representative point other than the measurement representative point is set around the correction target image range, and the amount of deviation at the auxiliary representative point is extrapolated from the amount of deviation measured at a nearby measurement representative point, and the correction target image range Are all divided into a triangle or a quadrangle having the measurement representative points or auxiliary representative points provided in the correction target image range as vertices, and the deviation amount in the triangle or the quadrangle is divided into the triangles. Alternatively, the image distortion correction method is characterized in that it is calculated by linear interpolation from a shift amount related to a vertex of a quadrangle.
[0014]
Furthermore, in the image distortion correction method of the present invention,
As the measurement representative points, a two-dimensional type that measures a two-dimensional deviation amount in two directions in an image, a vertical type that measures a one-dimensional deviation amount in a vertical direction, and a horizontal direction that measures a one-dimensional deviation amount in a horizontal direction. A measurement representative of any one of a direction type, a right-down diagonal direction type that measures a one-dimensional deviation amount in a right-down diagonal direction, and a right-up diagonal direction type that measures a one-dimensional deviation amount in a right-up diagonal direction Set the point
The distortion pattern in the entire image is determined by combining the deviation amounts at the measurement representative points of each type.
[0015]
Furthermore, in the image distortion correction method of the present invention,
The measurement representative points are automatically arranged based on the image characteristics of the local region of the reference image, determining the optimum position and type for pattern matching.
[0016]
Furthermore, in the image distortion correction method of the present invention,
Determine whether the result of pattern matching at the measurement representative point meets a predetermined standard,
The amount of deviation of the measurement representative point whose pattern matching result does not satisfy the predetermined standard is compensated from the amount of deviation at the measurement representative point near the measurement representative point,
Thereby, the distortion pattern of the whole image is determined.
[0017]
Furthermore, in the image distortion correction method of the present invention,
The distortion pattern of the entire image is expressed by a two-dimensional deviation amount for each small area of the image,
For each small area of the image, the two-dimensional deviation amount in the small area, the identification number of the measurement representative point or auxiliary representative point corresponding to the vertex of the triangle surrounding the center position of the small area, and the vertex of the triangle The interpolation coefficient for calculating the shift amount of the small area from the measured representative point or auxiliary representative point is held in association with each other. The identification number and interpolation coefficient of the measurement representative point or auxiliary representative point are calculated and set in advance when setting the measurement representative point or auxiliary representative point. The two-dimensional deviation amount for each small area can be calculated with a small amount of calculation.
[0018]
In order to achieve the object of the present invention, according to the first aspect of the present invention, a distortion pattern between a reference image and a comparative image is calculated by interpolation processing from a deviation amount measurement result by pattern matching at a measurement representative point. An image distortion correction apparatus is provided. The image distortion correction apparatus of the present invention isSet the measurement representative points corresponding to the reference image and the comparison image, andIn addition to the measurement representative points, auxiliary representative points are added to the periphery of the image and the four corners of the image. The present invention is characterized in that it is divided all over by a triangle composed of representative points or auxiliary representative points, and the deviation amount in the triangle is calculated from the deviation amounts of three measurement representative points by linear interpolation.
[0019]
In the image distortion correction apparatus of the present invention, the measurement representative point includes a two-dimensional measurement representative point for measuring a two-dimensional deviation amount, a vertical direction only, a horizontal direction only, a right-down diagonal direction, and a right-up diagonal direction. An arbitrary number of four types of one-dimensional representative points for measuring only one-dimensional deviation amounts are set, and deviation amounts at various representative points are combined to determine a distortion pattern in the entire image.
[0020]
Also, the correction distortion correction apparatus of the present invention determines the optimum position and representative point type for pattern matching based on the image features of the local region of the reference image and automatically arranges the measurement representative points.
[0021]
In addition, the correction distortion correction apparatus according to the present invention provides a luminance value fluctuation amount in the vicinity of the representative point of the reference image and the comparison image when the pattern matching at the representative point fails due to an image feature change between the reference image and the comparison image. The failure is detected based on the difference between the representative points of the reference image and the comparison image, and the amount of deviation of the representative points that failed the pattern matching is calculated at one to three measurement representative points near the corresponding representative point. After compensating for the shift amount, the distortion pattern of the entire image is determined.
[0022]
Furthermore, in the correction distortion correction apparatus of the present invention, the distortion pattern is held in the form of a deviation amount table in which the entire image is divided into small rectangles, and the two-dimensional deviation amount in each rectangle is a constant value. The amount of deviation at any of the positions can be quickly referred to by referring to the table, and the amount of deviation of each small rectangle from the three representative point numbers surrounding the center position of the rectangle and the three representative points is displayed in the amount of deviation table. An interpolation coefficient for calculation is attached as a deviation conversion table, the representative point number and the interpolation coefficient are calculated when the representative point is set, and the deviation amount table can be created with a small amount of calculation from the deviation amount of the representative point when correction is executed.
[0023]
DETAILED DESCRIPTION OF THE INVENTION
〔Constitution〕
The configuration of the image distortion correction system according to the first embodiment of the present invention is shown in FIG. The image distortion correction system according to the first embodiment of the present invention includes an image distortion correction apparatus 1, an image acquisition unit 2, an operation instruction unit 3, a result display unit 4, and a result storage unit 5. The image distortion correction apparatus 1 uses a reference image or a non-defective image received from the external image acquisition unit 2 to arrange a representative point arrangement processing unit 10 that arranges measurement representative points at positions appropriate for positional deviation measurement, and a representative point A displacement conversion table creation processing unit 20 for creating a table for converting a displacement amount into a displacement amount at each position of the image, and a displacement at a representative point in the comparison image or inspection image received from the image acquisition unit 2 are measured. And a distortion measurement processing unit 30 that creates a deviation amount table after compensating for the deviation amount of the measurement failure point. In addition, the image distortion correction apparatus 1 is accompanied by an internal storage unit 40 that stores representative point arrangement data 41, a shift conversion table 42, and a shift amount table 43.
[0024]
The image distortion correction apparatus 1 receives an operation instruction from the external operation instruction unit 3 and outputs a processing result to the external result display unit 4 and the result storage unit 5.
[0025]
[Representative point arrangement processing section]
FIG. 2 is an explanatory diagram of the representative point arrangement processing unit according to the first embodiment of the present invention. As a measurement representative point for measuring a deviation amount for determining a distortion pattern, for example, a two-dimensional measurement point for measuring a deviation amount in two orthogonal directions and a one-dimensional measurement for a deviation amount in one direction only in the vertical direction. Measurement point, one-dimensional measurement point for measuring the amount of deviation in one direction only in the horizontal direction, one-dimensional measurement point for measuring the amount of deviation in one direction only in the downward-right diagonal direction, and one direction in only the upward-right diagonal direction Four types of one-dimensional measurement points, which are one-dimensional measurement points for measuring the deviation amount, are used. As shown in FIG. 2, the representative point arrangement processing unit 10 includes an evaluation image creation unit 12 and a representative point position determination unit 14. In the evaluation image creation unit 12, based on the image characteristics of the local area of the non-defective image or the reference image, the two-dimensional arrangement evaluation image 131 and four types of one-dimensional arrangement evaluation images, that is, the vertical one-dimensional arrangement evaluation image 132, the horizontal direction A one-dimensional arrangement evaluation image 133, a right-down diagonal direction one-dimensional arrangement evaluation image 134, and a right-up diagonal direction one-dimensional arrangement evaluation image 135 are created. Thereby, for example, an image in which a high evaluation score is given to a pixel position where a specific local image feature exists is obtained. The representative point position determination unit 14 sequentially arranges measurement representative points at locations where the evaluation value of the image is high.
[0026]
When the measurement representative point is set, a non-defective image or a reference image in the deviation amount measurement area is cut out and stored as a position deviation measurement template. At this time, the standard deviation of the pixel value of the template image is also calculated and stored. Each positional deviation measurement representative point sets a search range corresponding to an assumed deviation amount. The representative point arrangement data 41 includes a representative point position, a positional deviation measurement template, a search range, and the like. [Displacement conversion table creation processing section]
[0027]
The shift conversion table 42 is a table for converting a two-dimensional shift amount at irregularly arranged measurement representative points into a two-dimensional shift amount of a predetermined small region obtained by dividing an image into small rectangles. The two-dimensional deviation amount in each small area is assumed to be constant. The shift conversion table 42 stores three representative point numbers surrounding the small rectangular center position and an interpolation coefficient for calculating the shift amount of the small rectangular center position from the shift amount of the three representative points by interpolation.
[0028]
FIG. 3 is an explanatory diagram of the shift conversion table creation processing unit 20. As shown in FIG. 3, the shift conversion table creation processing unit 20 adds auxiliary representative points (step 201), divides the entire image into triangles (step 202), and creates a table (step 203).
[0029]
The entire image can be divided by any polygon other than the triangle, but any polygon can be finally divided into triangles by overlapping the division. Therefore, in the embodiment of the present invention, a case where the entire image is divided into triangles will be described. In the following description, it is assumed that the correction target image range is a quadrangle.
[0030]
In the auxiliary representative point adding step 201, auxiliary representative points are added to the periphery of the image and the four corners of the image in addition to the measurement representative points for measuring the deviation, and the deviation amount at the auxiliary representative points is compensated from the deviation amount of the measurement representative point. Shall. In the entire image triangle dividing step 202, the entire image is divided into triangles having three representative points (that is, measurement representative points or auxiliary representative points) as vertices. In table creation step 203, three representative points constituting a triangle including the center of each small region are determined, and an interpolation coefficient for an interpolation operation for determining a shift amount of the small region center from the three representative points is calculated. To do.
[0031]
[Strain measurement processing section]
The operation of the distortion measurement processing unit 30 differs depending on the configuration of the measurement representative point type. The types of measurement representative points are the two-dimensional type that measures the two-dimensional displacement in two directions in the image, the vertical type that measures the one-dimensional displacement in the vertical direction, and the one-dimensional displacement in the horizontal direction. There are a horizontal direction type, a right-down oblique direction type that measures a one-dimensional deviation amount in a right-down oblique direction, and a right-up oblique direction type that measures a one-dimensional deviation amount in a right-up oblique direction.
[0032]
When all the measurement representative points are two-dimensional measurement representative points, the strain measurement processing unit 30 executes basic strain measurement processing. FIG. 4 is an explanatory diagram of the basic strain measurement process of the strain measurement processing unit 30 according to the first embodiment of the present invention. As shown in FIG. 4, the basic distortion measurement process includes a deviation measurement unit 31, a measurement failure deviation amount compensation unit 32, and a deviation amount table creation unit 33.
[0033]
The deviation measurement unit 31 performs pattern matching between an image near each deviation measurement representative point of the inspection image or the comparison image and the corresponding template, and measures the deviation amount.
[0034]
FIG. 5 is a process explanatory diagram of the measurement failure deviation amount compensation unit 32 according to the first embodiment of the present invention. The measurement failure deviation amount compensation unit 32 fails due to a difference in luminance value fluctuation amount between the measurement representative point neighboring images of the non-defective image or the reference image and the inspection image or the comparison image, and the degree of matching when the two images are matched. Is detected (step 321). The deviation amount of the measurement representative point that has failed the pattern matching is compensated by interpolation using the same method as the deviation conversion table from the deviation amounts at the three measurement representative points in the vicinity of the measurement representative point (step 322).
[0035]
The deviation amount table creation unit 33 compensates the deviation amount at the auxiliary representative point from the corresponding measurement representative point, and creates a deviation amount table 43 representing the deviation amount of each small rectangle using the deviation conversion table 42.
[0036]
FIG. 6 is an explanatory diagram of a two-dimensional / one-dimensional mixed strain measurement process according to the first embodiment of the present invention.
[0037]
When both the two-dimensional measurement representative point and the one-dimensional measurement representative point are set in the image, the deviation amount of the two-dimensional measurement representative point is first measured as shown in FIG. 6 (step 301), and this two-dimensional measurement is performed. A distortion pattern is provisionally determined from the amount of deviation of the representative point (step 303). Based on this result, the position of the one-dimensional measurement representative point is moved (step 304), and deviation measurement is performed by pattern matching using the movement destination as a base point (step 305). Finally, the measurement results at all measurement points are collected and a deviation amount table is created (step 306).
[0038]
FIG. 7 is a distortion measurement processing flowchart of one-dimensional measurement of the distortion measurement processing unit according to the first embodiment of the present invention. When there is no two-dimensional measurement representative point and only the one-dimensional measurement representative point is set, as shown in FIG. 7, first, the deviation measurement of all the one-dimensional measurement representative points is performed once (step 311), and the predetermined number of times is reached. It is determined whether or not (step 313), and if the predetermined number of times has not been reached, the following processing is repeated an arbitrary number of times. For each type of one-dimensional measurement point, a distortion pattern is provisionally determined using only one-dimensional measurement representative points other than the same type (step 315), and based on this result, the position of this type of one-dimensional measurement representative point is moved. (Step 316), displacement measurement is performed by pattern matching using the movement destination as a base point (Step 311). Finally, the measurement results at all measurement points are collected and a deviation amount table is created (step 314).
[0039]
According to the image distortion correction system of the first embodiment of the present invention as described above, it is possible to automatically set a deviation measurement representative point even for irregular image distortion, and a deviation conversion table when setting the measurement representative point. Therefore, it is possible to quickly determine a distortion pattern and perform distortion correction processing on a new inspection image or comparison image.
[0040]
As a result, in the conventional technology, it was difficult due to image distortion, and the image was taken using a line scan camera in a state where there was unevenness in the appearance inspection of the electronic circuit configured on the flexible base material and the speed of the conveyance system. Inspection and identification processing based on images can also be realized within a processing time that is practically acceptable.
[0041]
Further, according to the image distortion correction system of the first embodiment of the present invention, since the distortion pattern of the entire image can be determined by combining the deviation information at the two-dimensional measurement point and the one-dimensional measurement point, Even when there are few places suitable for measurement, image distortion measurement can be performed.
[0042]
According to the image distortion correction system of the first embodiment of the present invention, even if the measurement representative point of the inspection image or the comparison image has a defect or is blocked by a foreign substance, the failure detection is detected. However, it is possible to compensate for the amount of deviation from the surrounding measurement points, and to perform image distortion measurement that is not easily affected by defects and foreign matter.
[0043]
Hereinafter, a second embodiment of the present invention in which the present invention is applied to a printed circuit board pattern inspection apparatus configured on a flexible base material will be described.
[0044]
FIG. 8 is a schematic configuration diagram of a printed circuit board pattern inspection system according to a second embodiment of the present invention. The printed circuit board pattern inspection system of this embodiment includes a pattern inspection apparatus 101, a camera 102, an external input device 103, and an external output device 104. The external input device 103 inputs an instruction to perform learning or inspection from the operator. The external output device 104 outputs the status of the processing device and the inspection result, and shows it to the operator. The camera 102 captures a printed circuit board pattern to be inspected and transmits a video signal to the pattern inspection apparatus 101.
[0045]
The pattern inspection apparatus 101 controls imaging by the camera 102, converts a video signal from the camera 102 into a digital image, a main processing system 106 including a distortion correction processing system 109 and an inspection processing system 110, The internal storage unit 40 includes an external input device 103 and an input / output control system 108 connected to the external output device 104. In the case of a single image distortion correction apparatus, the main processing system 106 is composed of only the distortion correction processing system 109.
[0046]
FIG. 9 is a data flow diagram showing a data flow of the printed circuit board pattern inspection system according to the second embodiment of the present invention. As shown in the figure, the distortion correction processing system 109 includes the representative point arrangement processing unit 10, the deviation conversion table creation processing unit 20, and the distortion measurement processing unit 30 described with respect to the first embodiment of the present invention. include.
[0047]
The video signal captured from the camera 102 is supplied to the distortion correction processing system 109 in the form of image data via the camera control system 105. The image data and the deviation table processed by the distortion correction processing system 109 are supplied to the inspection processing form 110. The distortion correction data generated by the distortion correction processing system 109 is stored in the internal storage unit 40.
[0048]
Numerical data or image data generated by the distortion correction processing system 109 and the inspection processing system 110 is supplied to the input / output control system 108 and presented to the user via the external output device 104. Alternatively, input data from the user is given from the external input device 103 to the distortion correction processing system 109 and the inspection processing system 110 via the input / output control system 108.
[0049]
The representative point arrangement processing unit 10 uses the reference image or the non-defective image to arrange the measurement representative points at a position appropriate for the positional deviation measurement, and the deviation conversion table creation processing unit 20 determines the positional deviation amount of the representative point of the image. A table to be converted into a deviation amount at each position is created, and the distortion measurement processing unit 30 measures a positional deviation at a representative point in the inspection image or the comparison image and compensates for the deviation amount at the measurement failure point, and then the deviation amount. Create a table. The representative point arrangement data and the deviation conversion table are stored in the internal storage unit 40.
[0050]
FIG. 10 is a process flowchart showing a process flow of the representative point arrangement processing unit 10 of the distortion correction processing system 109 in the pattern inspection apparatus 101 according to the second embodiment of the present invention.
[0051]
The representative point arrangement processing unit 10 captures a non-defective printed circuit board pattern with a camera, inputs non-defective image data (step 11), and creates an evaluation image for representative point arrangement based on the characteristics of the local region of the non-defective product image ( Step 12).
[0052]
Five types of evaluation images corresponding to four types of one-dimensional measurement points, that is, two-dimensional measurement points, that is, only in the vertical direction, only in the horizontal direction, only in the diagonally downward direction, and only in the diagonally upward direction, are created. FIG. 11 shows a flowchart of an evaluation image creation process according to the second embodiment of the present invention.
[0053]
For the evaluation image of the one-dimensional measurement point, for example, first, the image is differentiated in the direction in which the deviation measurement is performed (steps 21-1, 21-2, 21-3, or 21-4), and then the deviation measurement is performed. Integration is performed with an integration length that is about the size of the area to be arranged along the direction orthogonal to the direction of performing (steps 22-1, 22-2, 22-3, or 22-4). In this way, the evaluation value of the portion where the edge continues about the area size along the direction orthogonal to the measurement direction becomes high.
[0054]
Two-dimensional measurement point evaluation images include two types of orthogonal two-dimensional one-dimensional measurement point evaluation image sets (that is, a set of vertical and horizontal evaluation images, and a right-down diagonal direction and a right-up diagonal direction). Set) minimum value images are created (steps 23-1 and 23-2), and a maximum value image is created from the minimum value images (step 24). Thereby, the evaluation value of the part which has an edge component about both directions becomes high.
[0055]
Here, the minimum value image of the set of images is an image in which the value at the pixel position (p) of the minimum value image is the minimum value of the values at the corresponding pixel position (p) of each image of the set of images. is there. The maximum value image of the set of images is an image in which the value at a pixel position (p) of the maximum value image is the maximum value at the corresponding pixel position (p) of each image of the set of images. is there. By applying such minimum value imaging and maximum value imaging to four types of one-dimensional layout evaluation images, a two-dimensional layout evaluation image can be obtained.
[0056]
More specifically, first, four types of one-dimensional arrangement evaluation images are differentiated in corresponding directions, and then integrated along a direction orthogonal to the directions. For example, a vertical one-dimensional layout evaluation image is created by differentiating a non-defective image in the vertical direction and then integrating in the horizontal direction. Differentiating in the vertical direction (more precisely, taking the absolute value by differentiating) increases the evaluation of the edge part in the horizontal direction, and integrating this in the horizontal direction evaluates the part where the edge is continuous in the horizontal direction. Becomes higher.
[0057]
Next, minimum value images in two orthogonal directions (vertical direction and horizontal direction, or upward and downward direction) are created. In this way, for example, when taking the minimum value in the vertical direction and the horizontal direction, the evaluation is high only at the portion where the edge is continuous in both the vertical direction and the horizontal direction (the evaluation is high because there is an edge in the vertical direction). If there is no side edge and the evaluation is 0, the minimum value image is 0).
[0058]
Finally, maximum value images of “minimum value images in the vertical and horizontal directions” and “minimum value images in the upward and downward directions” are created. This maximum value image has a high evaluation of “there are edges in both the vertical direction and the horizontal direction” or “the edges are in both the upward and downward directions”.
[0059]
FIG. 12 is a flowchart of representative point arrangement processing according to the second embodiment of the present invention. For representative point arrangement, a representative point area size, a shift search range, an arrangement interval, and an evaluation value lower limit are set in advance, and a maximum evaluation position search is performed (step 31). The maximum evaluation position search includes a vertical one-dimensional layout evaluation image, a horizontal one-dimensional layout evaluation image, a right-down diagonal direction one-dimensional layout evaluation image, a right-up diagonal direction one-dimensional layout evaluation image, and a two-dimensional layout evaluation image. A location where the evaluation value is maximized is searched with five types of evaluation images. If the obtained maximum evaluation value is greater than or equal to the lower limit (step 32), the location is registered as a representative point (step 33). When registering the representative point, the representative point area size and the shift amount search range are recorded together with the representative point position. When one representative point is arranged, all the evaluation values of the circular area whose radius is the arrangement interval around this point are all set to 0 (step 33), and the process returns to step 31 to search for the next representative point. When the maximum evaluation value is less than the lower limit of the evaluation value (step 32), the representative point arrangement process is terminated.
[0060]
Instead of evaluating the five types of evaluation images separately, each can be multiplied by a weighting factor, and one maximum value image can be created and used alone for placement. In this case, it is possible to control which of the two-dimensional representative points and the one-dimensional representative points are arranged more by the weight.
[0061]
FIG. 13 is a process flow diagram showing a process flow of the shift conversion table creation processing unit 20 of the distortion correction processing system 109 in the pattern inspection apparatus according to the second embodiment of the present invention.
[0062]
In step 41, for example, about the square root of the number of measurement representative pointsNumberAn auxiliary representative point is added to each side of the image periphery. As an arrangement method, for each side of the image periphery,Near the edgeAn auxiliary representative point is arranged at a position where a perpendicular is dropped from the measurement representative point to the corresponding side. The amount of deviation of this auxiliary representative point is the same as the corresponding measurement representative point. Furthermore, auxiliary representative points are also added to the corners of the periphery of the image (for example, the four corners of the image when the correction target image region is a quadrangle). The shift amount of the auxiliary representative points arranged at the corners of the image is the average of the two auxiliary representative points that are present on both sides to be connected and are closest to the corners.
[0063]
Next, the entire image is divided into triangles (step 42). FIG. 14 is a flowchart of this image triangulation process. In the triangular division of the image, for example, first, the side connecting the auxiliary measurement points arranged on the four sides around the image is determined (step 51). Next, an incomplete edge is searched until a completion condition that a triangle is generated on one side with respect to the edges around the image and a triangle is generated on both sides with respect to the edges inside the image is satisfied ( In step 52), if there is an incomplete edge in step 53, the process proceeds to step 54, a vertex corresponding to the incomplete edge is searched for to generate an edge, and if the search is completed, the triangle dividing process is terminated. .
[0064]
In step 54, one vertex is sequentially determined for one side. At this time, the condition that the distance to the current side of interest is as close as possible, the interior angles on both sides of the current side are as equal as possible, no other vertices are included in the created triangle, and the sides do not intersect To determine the vertices sequentially and generate edges (ie, determine the triangle).
[0065]
FIG. 15 is a diagram illustrating an example of a triangular division result of an image. In FIG. 15, the direction in which a perpendicular is drawn from the measurement representative point to each side in order to generate the auxiliary representative point, and the shift amount compensation relationship at the auxiliary representative point are indicated by arrows. The correction area corresponding to the measurement representative point is surrounded by a solid line, and the auxiliary area corresponding to the auxiliary representative point is surrounded by a dotted line.
[0066]
The distortion pattern of the entire image is represented by a deviation amount table that accommodates a deviation amount corresponding to each small rectangle obtained by dividing the image into small rectangles, and the deviation amount in the small rectangle is constant. The deviation amount table associates the representative point (measurement representative point or auxiliary representative point) number constituting the triangle including the center of each small rectangle with the coefficient for linearly interpolating the deviation amount from the three representative points. The shift conversion table is formed.
[0067]
FIG. 16 is a diagram for explaining the relationship between the triangle and the center of the small rectangle in the second embodiment of the present invention. As shown in the figure, the linear interpolation of the deviation amount is defined as ABC for the triangle, P for the center of the small rectangle, Q for the intersection of the straight line passing through AP and BC,
Amount of deviation at A (dxa, dya)
Deviation amount at B (dxb, dyb)
Deviation amount at C (dxc, dyc)
BQ: QC = t: (1−t)
AP: PQ = u: (1−u)
Then, the deviation amount at P is
dxp = dxa x (1-u) + (dxb x (1-t) + dxc x t) x u
dyp = dya × (1−u) + (dyb × (1−t) + dyc × t) × u
It can be expressed as
[0068]
Linear interpolation by three points corresponds to expressing a deviation amount distribution in a triangle by a plane composed of three points when considering a three-dimensional space in which an image plane is an XY plane and a deviation is a Z axis. In addition, the interpolation coefficient can be calculated at the time of measurement point arrangement, and when performing distortion correction processing on an inspection image, interpolation calculation can be performed with a slight product-sum calculation. Therefore, this is a method that can most effectively utilize the deviation information of the three points without increasing the processing speed.
[0069]
FIG. 17 is a process flow diagram showing a process flow of the distortion measurement processing unit 30 of the distortion correction processing system 109 in the printed circuit board pattern inspection system according to the second embodiment of the present invention. Here, it is assumed that all five types of measurement representative points are set. First, the deviation amount of the two-dimensional measurement representative point is measured by pattern matching based on the normalized correlation (step 61), and the deviation amount compensation process of the measurement failure representative point is performed (step 62). A pattern is provisionally determined (step 63). Based on this result, the position of the one-dimensional measurement representative point is moved (step 64), deviation measurement by pattern matching using the movement destination as a base point (step 65), and deviation amount compensation processing of the measurement failure representative point (step 66). I do. Further, for each type of one-dimensional measurement point, a distortion pattern is provisionally determined at all measurement representative points other than the same type (step 68), and the position of the corresponding one-dimensional measurement representative point is moved based on the result (step 68). 59) Deviation measurement by pattern matching using the movement destination as a base point (step 65) and deviation amount compensation processing of the measurement failure representative point (step 66) are performed, and this is repeated an arbitrary number of times (step 67).
[0070]
In addition, the failure detection in the measurement failure representative point deviation amount compensation process calculates the standard deviation of the representative point corresponding area image at the maximum matching position of the inspection image, and this is constant compared to the standard deviation of the calculated good product image. If it is smaller than the ratio, it is considered as a failure. A failure also occurs when the correlation coefficient at the maximum matching position falls below a certain value. In the failure compensation process, three representative points related to the creation of the same deviation amount table in the vicinity of the failure measurement are determined in advance by the same method as the deviation conversion table creation unit, and are determined from these three points by linear interpolation.
[0071]
Finally, an apparatus for simultaneously correcting misalignment and distortion according to a third embodiment of the present invention will be described. FIG. 18 is an explanatory diagram of the positional deviation and distortion separation measurement and integration processing according to the third embodiment of the present invention. For example, the pattern inspection apparatus according to the second embodiment of the present invention is simple when the image to be inspected is misaligned due to translation and rotation of the entire image, but the local distortion amount is smaller than the misalignment amount. It is possible to provide a device that can simultaneously correct misalignment and distortion by a simple method.
[0072]
That is, as shown in FIG. 18, first, the positional deviation of the one-dimensional measuring point or the two-dimensional measuring point for measuring the positional deviation is measured (step 71), two one-dimensional measuring points or one two-dimensional A parallel position shift due to the measurement points or a parallel shift and rotation position shift between the two two-dimensional measurement points is determined, and a position shift table is created (step 72). Then, the measurement representative point position for distortion correction is moved (step 73), and the distortion amount is measured (step 74). Finally, the sum of positional deviation and distortion deviation is stored in the integrated deviation table (step 75). In this way, the search range for distortion correction can be reduced, and the shift amount can be referred to from a single table, so that the inspection process can be speeded up.
[0073]
The image distortion correction method according to the embodiment of the present invention can be constructed by software (program), and the image distortion correction apparatus according to the embodiment of the present invention is realized by executing this program by the CPU of the computer. can do. The built program is recorded in a disk device or the like and installed in a computer as necessary, and stored in a portable recording medium such as a flexible disk, a memory card, or a CD-ROM, and installed in the computer as needed. Alternatively, it is installed in a computer via a communication line or the like and executed by the CPU of the computer.
[0074]
The present invention is not limited to the above-described embodiments, and various modifications and applications can be made within the scope of the claims.
[0075]
【The invention's effect】
  According to the present invention,A distortion pattern that smoothly changes in the entire correction target image can be generated from the measurement representative points arranged in the correction target image in a short processing time.. In addition, measurement representative points can be automatically placed at appropriate locations,Distortion correction can be performed when a location suitable for two-dimensional deviation measurement is limited in the image, or even if measurement fails at some measurement representative points.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of an image distortion correction system according to a first embodiment of the present invention.
FIG. 2 is an explanatory diagram of a representative point arrangement processing unit according to the first embodiment of the present invention.
FIG. 3 is an explanatory diagram of a shift conversion table creation processing unit according to the first embodiment of the present invention.
FIG. 4 is an explanatory diagram of basic distortion measurement processing according to the first embodiment of the present invention.
FIG. 5 is an explanatory diagram of a measurement failure deviation amount compensation unit according to the first embodiment of the present invention.
FIG. 6 is an explanatory diagram of a two-dimensional / one-dimensional mixed strain measurement process according to the first embodiment of the present invention.
FIG. 7 is a flowchart of one-dimensional distortion measurement processing according to the first embodiment of the present invention.
FIG. 8 is a configuration diagram of a printed circuit board pattern inspection system according to a second embodiment of the present invention.
FIG. 9 is a data flow diagram of a printed circuit board pattern inspection system according to a second embodiment of the present invention.
FIG. 10 is a process flow diagram of a representative point arrangement processing unit according to the second embodiment of the present invention.
FIG. 11 is a flowchart of an evaluation image creation process according to the second embodiment of the present invention.
FIG. 12 is a flowchart of representative point arrangement processing according to the second embodiment of the present invention.
FIG. 13 is a flowchart of a shift conversion table creation process according to the second embodiment of the present invention.
FIG. 14 is a flowchart of image triangle division processing according to the second embodiment of the present invention;
FIG. 15 is an explanatory diagram of an example of a triangular division result of an image.
FIG. 16 is an explanatory diagram of a triangle and a center of a small rectangle in the second embodiment of the present invention.
FIG. 17 is a flowchart of distortion measurement processing according to the second embodiment of the present invention.
FIG. 18 is an explanatory diagram of the positional deviation and distortion separation measurement and integration processing according to the third embodiment of the present invention.
[Explanation of symbols]
1 Image distortion correction device
2 Image acquisition unit
3 Operation instruction section
4 results display
5 result accumulation part
10 Representative point placement processing section
11 Representative point placement section
20 Deviation conversion table creation processing section
21 Deviation conversion table creation section
30 Strain measurement processing unit
31 Representative point position measurement unit
32 Measurement failure deviation compensation unit
33 Deviation table creation unit
40 Internal storage
41 Representative point arrangement data
42 Deviation conversion table
43 Deviation table

Claims (15)

An image distortion correction method for calculating a distortion pattern of a comparison image with respect to a reference image by interpolation processing from a deviation amount measured by pattern matching at a corresponding measurement representative point,
Set the corresponding measurement representative points of the reference image and the comparative image, further wherein the set of auxiliary representative points other than the measurement representative points around the correction target image area,
The amount of deviation at the auxiliary representative point is extrapolated from the amount of deviation measured at the nearby measurement representative point,
Dividing the entire region of the correction target image range into a triangle or a quadrangle with the measurement representative points or auxiliary representative points provided in the correction target image range as vertices;
The amount of deviation within the triangle or quadrangle is calculated by linear interpolation from the amount of deviation associated with the apex of the triangle or quadrangle.
An image distortion correction method characterized by the above. As measurement representative points, a two-dimensional type that measures a two-dimensional displacement amount in two directions in an image, a vertical type that measures a one-dimensional displacement amount in a vertical direction, and a horizontal direction that measures a one-dimensional displacement amount in a horizontal direction. Measurement representative point of any of the following types: a right-down diagonal direction type that measures the one-dimensional deviation amount in the right-down diagonal direction and a right-up diagonal direction type that measures the one-dimensional deviation amount in the right-up diagonal direction Set
Combine the amount of deviation at each type of measurement representative point to determine the distortion pattern in the entire image,
The image distortion correction method according to claim 1.   The image distortion correction method according to claim 1, wherein the measurement representative point is determined in position and type based on an image feature of a local region of the reference image. Determine whether the result of pattern matching at the measurement representative point meets a predetermined standard,
The amount of deviation of the measurement representative point whose pattern matching result does not satisfy the predetermined standard is compensated from the amount of deviation at the measurement representative point near the measurement representative point.
The image distortion correction method according to claim 1. The distortion pattern of the entire image is expressed by a two-dimensional deviation amount for each small area of the image,
For each small area of the image, the two-dimensional deviation amount in the small area, the identification number of the measurement representative point or auxiliary representative point corresponding to the apex of the triangle or quadrangle surrounding the center position of the small area, the triangle or An interpolation coefficient for calculating the amount of deviation of the small area from the measurement representative point or auxiliary representative point corresponding to the vertex of the quadrangle,
The image distortion correction method according to claim 1. An image distortion correction apparatus for calculating a distortion pattern between a reference image and a comparison image by interpolation processing from a deviation amount measurement result by pattern matching at a measurement representative point,
Set the measurement representative points corresponding to the reference image and the comparison image, and
Means for setting an auxiliary representative point other than the measurement representative point around the correction target image range, and extrapolating the deviation amount at the auxiliary representative point from the deviation amount measured at the nearby measurement representative point;
Means for dividing the entire region of the correction target image range into a triangle or a quadrangle with the measurement representative points or auxiliary representative points provided in the correction target image range as vertices;
Means for calculating a deviation amount in the triangle or the quadrangle by linear interpolation from a deviation amount related to the vertex of the triangle or the quadrangle;
An image distortion correction apparatus comprising: A two-dimensional measurement representative point for measuring a two-dimensional deviation amount, a first one-dimensional measurement representative point for measuring a one-dimensional deviation amount only in the vertical direction, and a second one-dimensional measurement for measuring a one-dimensional deviation amount only in the horizontal direction. Of the representative point, the third one-dimensional measurement representative point that measures the one-dimensional deviation amount only in the diagonally downward direction, and the fourth one-dimensional measurement representative point that measures the one-dimensional deviation amount only in the diagonally upward direction Means for setting one of the types of measurement representative points,
Means for determining the distortion pattern in the entire image by combining the amount of deviation at each type of measurement representative point;
The image distortion correction apparatus according to claim 6, further comprising: The means for setting any type of measurement representative point determines and arranges the position and type of the measurement representative point optimal for pattern matching based on the image characteristics of the local region of the reference image.
The image distortion correction apparatus according to claim 7. Means for determining whether the result of pattern matching at the measurement representative point satisfies a predetermined criterion;
Means for compensating the deviation amount of the measurement representative point whose pattern matching result does not satisfy the predetermined standard from the deviation amount at another measurement representative point in the vicinity of the measurement representative point;
The image distortion correction apparatus according to claim 6, further comprising: For each small area of the image, a two-dimensional deviation amount in the small area representing the distortion pattern of the entire image,
An identification number of a measurement representative point or auxiliary representative point corresponding to a vertex of a triangle or a quadrangle surrounding the center position of the small area;
An interpolation coefficient for calculating the shift amount of the small region from the measurement representative point or auxiliary representative point corresponding to the triangle or the apex of the quadrangle;
The image distortion correction apparatus according to claim 6, further comprising a storage unit that stores the information in association with each other. An image distortion correction program for calculating a distortion pattern of a comparison image with respect to a reference image by interpolation processing from a deviation amount measured by pattern matching at a corresponding measurement representative point,
A function for setting a measurement representative point corresponding to the reference image and the comparison image, and further setting an auxiliary representative point other than the measurement representative point around the correction target image range;
A function of extrapolating the amount of deviation at the auxiliary representative point from the amount of deviation measured at a nearby measurement representative point;
A function to divide the entire area of the correction target image range into a triangle or a quadrangle with the measurement representative points or auxiliary representative points provided in the correction target image range as vertices;
A function of calculating a deviation amount in a triangle or a quadrangle by linear interpolation from a deviation amount related to the apex of the triangle or the quadrangle;
An image distortion correction program for realizing a computer. As measurement representative points, a two-dimensional type that measures a two-dimensional displacement amount in two directions in an image, a vertical type that measures a one-dimensional displacement amount in a vertical direction, and a horizontal direction that measures a one-dimensional displacement amount in a horizontal direction. Measurement representative point of any of the following types: a right-down diagonal direction type that measures the one-dimensional deviation amount in the right-down diagonal direction and a right-up diagonal direction type that measures the one-dimensional deviation amount in the right-up diagonal direction A function to set
A function that determines the distortion pattern in the entire image by combining the amount of deviation at each type of measurement representative point,
The image distortion correction program according to claim 11, further causing the computer to realize the above.   The image distortion correction program according to claim 10 or 12, further causing a computer to realize a function of determining a position and type of a measurement representative point based on an image feature of a local region of a reference image. A function for determining whether the result of pattern matching at the measurement representative point satisfies a predetermined standard;
A function of compensating the deviation amount of the measurement representative point whose pattern matching result does not satisfy the predetermined standard from the deviation amount at the measurement representative point in the vicinity of the measurement representative point;
14. The image distortion correction program according to claim 11, wherein the image distortion correction program is further implemented by a computer. For each small area of the image, a two-dimensional deviation amount in the small area representing the distortion pattern of the entire image,
The identification number of the measurement representative point or auxiliary representative point corresponding to the vertex of the triangle or quadrangle surrounding the center position of the small area, and the measurement representative point or auxiliary representative point corresponding to the vertex of the triangle or quadrilateral A function of associating and holding an interpolation coefficient for calculating a shift amount of a region;
15. The image distortion correction program according to claim 10, further comprising:

Images