JP4611708B2 - High-precision mark position / posture detection device - Google Patents

Description

  The present invention improves the assembly accuracy of components, for example, when an electronic component such as a semiconductor is accurately placed on a printed circuit board by a robot or a plurality of IC chips are placed on a single IC package by a robot. Therefore, the present invention relates to a high-accuracy mark position / orientation detection device that accurately detects the position / orientation of a mark pre-written on a part.

  2. Description of the Related Art Conventionally, image measurement is performed in which a mark is photographed using a camera for position alignment, that is, component alignment, for accurately placing an electronic component on a printed circuit board, and the image is analyzed. In this image measurement, a high-resolution camera having about 1.5 million pixels is used to improve the measurement accuracy of the position / orientation of the component, so the cost of the image measurement device has increased.

  In order to reduce the cost of the image measuring device, an image processing technique with an accuracy of less than the pixel resolution (sub-pixel) is indispensable for a low-resolution camera image of about 310,000 pixels.

  As a method for detecting a specific mark (pattern) from an input image, template matching based on image correlation is generally used. For example, a printed circuit board on which an IC component is placed, that is, an image of a mark provided on the workpiece side is held in advance as a template, and a mark portion extracted by scanning an image taken with a camera is compared. It is. In this case, for example, a template in which the mark is rotated once every time and a size whose size is changed every 1%, for example, are prepared. And it compares with the mark image extracted from the mark.

  At this time, the mark image and each mark of the template are compared, and the position / posture of the workpiece is obtained by moving one pixel at a time and searching for the smallest difference.

  For this reason, since template matching requires a long time in order to accurately determine the position and orientation, high-speed processing by hardware has also been realized in recent years (see, for example, Patent Document 1).

There is also a method of estimating from a similarity distribution obtained by template matching (see, for example, Patent Document 2).
JP-A-8-147477 After specifying the approximate position of the mark by such a method, the mark position is measured with sub-pixel accuracy.

  Conventionally, as a sub-pixel processing technique, a curved surface is applied to the similarity obtained by template matching, that is, a distribution of image correlation values, and a peak position is estimated by sub-pixels. A method is used in which points are measured by subpixels by function approximation of luminance change and estimated by fitting a straight line to these measurement points.

  However, in the method of estimating the peak position of the image correlation value distribution, there is a problem that a deviation is likely to occur between the peak position of the image correlation value distribution and the mark position due to the influence of image noise. There is a problem that measurement accuracy cannot be obtained.

  Even with edge position measurement and straight line fitting, if the known shape of the workpiece or mark is not used as a constraint, sufficient measurement accuracy cannot be obtained, and the known shape of the workpiece or mark is used as a constraint. In this case, there is a problem in that enormous processing time is required because iterative calculation is required.

  Since it is indispensable to take a large number of measurement points on the edge for accurate line fitting, there is a problem in that the amount of image measurement processing is large.

  The values necessary for determining the position of the workpiece are the coordinate positions Xm and Ym of the center in the two-dimensional coordinates XY of the pattern and the inclination of the pattern, that is, the rotation angle θ indicating the posture.

  In the present invention, for example, an image obtained by providing a mark with a known shape on the screen having two or more sets of parallel sides such as a quadrangular shape, such as a rectangle, on the screen. The posture of the mark, that is, the rotation angle θ, that minimizes the sum of squares of errors between the measurement points on all the parallel sides on the mark obtained by processing and the known positional relationship of all the parallel sides is obtained.

  Next, in each set of parallel sides of the mark, the sum of squares of the position error in the width direction of the parallel sides and the measurement points on all the parallel sides on the mark obtained by image processing of the image of the photographed mark with the camera is obtained. The positions Xm and Ym of the parallel sides that are minimized are obtained using the rotation angle θ.

  According to the present invention, the following effects can be obtained.

  (1) The number of measurement points on the side to be measured may be any number and is easy to measure. Also, I don't care about the order or interval. Since it can be solved analytically, calculations can be easily solved. Since there are as many samples as the number of combinations of measurement points on each side, an optimal solution can be obtained with at least the number of measurement points, and the accuracy is improved. Further, when the rotation angle θ is obtained, template matching can be easily performed by rotating the template accordingly, and the processing in image processing becomes easy.

  (2) When the rotation angle θ is obtained, it is possible to use the constraint condition including the width by applying the condition that the rotation angle θ is parallel and the width value, and the measurement accuracy can be improved.

  In the present invention, an image obtained by photographing a work on which a mark is formed with a television camera is input, and a rough template position is prepared by template prepared by rotating and enlarging / reducing the mark at a plurality of angles. To detect.

  Thereby, the approximate center position of the mark is detected, and the edge point of the mark is detected at a predetermined angle. Then, edge points are measured by polar coordinate conversion.

  Thus, for a mark having two or more sets of different parallel sides, such as a rectangle, such as a rectangle, the error between the measurement point on all parallel sides on the mark and the known positional relationship of all parallel sides is 2 The posture θ of the mark that minimizes the sum of products is obtained with an accuracy equal to or lower than the pixel resolution.

  Next, for the marks having two or more sets of parallel sides different in direction, the sum of squares of the position errors in the width direction of the measurement points on the parallel sides and the width direction of the parallel sides is minimum in each set of parallel sides. By obtaining the position of the parallel side, the position of the mark can be obtained with an accuracy equal to or lower than the pixel resolution.

  Before describing the embodiment of the present invention, a method for obtaining the inclination of the mark and the position of the mark will be described.

As shown in FIG. 7, when the length of the perpendicular line from the origin of the straight line L ₀ having an inclination of the x-axis and θ is D, each point on the straight line L ₀ is obtained by the following equation 1 indicating a straight line equation. .

本発明では、例えば矩形、平行四辺形の如き４角形、十字形等の２組以上の方向の異なる平行な辺をもつマークに対して、マーク上の全平行辺上の既知の位置関係との誤差の２乗和が最小となるマークの姿勢、つまり回転角θを画素分解能以下の精度で求める。

In the present invention, for example, a mark having two or more sets of parallel sides in different directions such as a rectangle, a quadrangle such as a parallelogram, a cross, etc., and a known positional relationship on all parallel sides on the mark. The posture of the mark that minimizes the sum of squared errors, that is, the rotation angle θ is obtained with an accuracy equal to or less than the pixel resolution.

As shown in FIG. 1, each side of the mark M is L ₁ , L ₂ , L ₃ , L ₄ , L ₁ and L ₃ are parallel, and L ₂ and L ₄ are parallel in different directions. Now each P ₁₁ to P _1i measurement points of the points on these _{sides, P 21 ~P 2j, P 31} ~P 3k, and P ₄₁ to P _4l. Note each of the measurement points _{P 11 ~P 1i, P 21 ~P} 2j, P 31 ~P 3k, the coordinates of _{P 41 ~P 4l P 11 (X} 11, Y 11) ~P 1i (X 1 i, Y 1i) _{, P 21 (X 21, Y} 21) ~P 2j (X 1j, Y 2j), P 31 (X 31, Y 31) ~P 3k (X 3 K, Y 3k), P 41 (X 41, Y 41 ) To P _4l (X _4l , Y _4l ).

The distances between the sides L ₁ , L ₂ , L ₃ and L ₄ and the origin 0 are D ₁ , D ₂ , D ₃ and D ₄ , respectively. When the angle formed by the pair of parallel sides is φ and the inclination of the mark M is θ, the following constraint formulas are obtained with respect to the measurement points on each side.

About side L ₁ of mark M

マークＭの辺Ｌ₂ については、Ｌ₁ をφだけ傾けたものであるから、

For the side L ₂ of the mark M, L ₁ is inclined by φ.

マークＭの辺Ｌ₃ については、Ｌ₁ を１８０度回転したものであるから、

Since the side L ₃ of the mark M is obtained by rotating L ₁ by 180 degrees,

マークＭの辺Ｌ₄ については、Ｌ₃ とφだけ傾けたものであるから、

Since the side L ₄ of the mark M is inclined by L ₃ and φ,

以上の関係式と、マークＭの幅つまり平行線Ｌ₂ とＬ₄ の幅Ｗと、マークＭの高さつまり平行線Ｌ₁ とＬ₃ の幅Ｈが既知であり、下記の数６に示す関係から、

The above relational expression, the width of the mark M, that is, the width W of the parallel lines L ₂ and L ₄ , and the height of the mark M, that is, the width H of the parallel lines L ₁ and L ₃ are known. From the relationship

以下の条件式が得られる。

The following conditional expression is obtained.

  For the width W of the mark M, the conditional expression shown in Equation 7 is obtained.

マークＭの高さＨについて

About the height H of the mark M

以上の条件式から、以下の数９が得られる。

From the above conditional expression, the following formula 9 is obtained.

従ってマークＭの姿勢θの最適推定値は、数１０で示す、以下の評価関数Ｊを最小にするθとして得られる。

Therefore, the optimum estimated value of the posture θ of the mark M is obtained as θ that minimizes the following evaluation function J, which is expressed by Equation 10.

このＪを最小にするθ^* については解析解が存在し、以下の数１１のように容易に解を得ることができる。

There is an analytical solution for θ ^* that minimizes J, and a solution can be easily obtained as shown in Equation 11 below.

ただしＡ，Ｂは数１２に示す通りである。

However, A and B are as shown in Equation 12.

次に、前記の如く、例えば矩形、平行四辺形の如き４角形、十字形などの２組以上の方向の異なる平行な辺をもつマークに対して、マークの姿勢つまり回転角θが得られた場合に、各平行辺の組において、平行辺上の計測点と平行辺の幅方向の位置誤差の２乗和が最小となる平行辺の位置を求めることで、マークの位置を画素分解能以下の精度で求める。

Next, as described above, the posture of the mark, that is, the rotation angle θ is obtained for a mark having two or more sets of parallel sides such as a rectangle, a quadrangle such as a parallelogram, and a cross. In this case, in each pair of parallel sides, the position of the mark is less than or equal to the pixel resolution by obtaining the position of the parallel side that minimizes the sum of squares of the measurement points on the parallel side and the position error in the width direction of the parallel side. Calculate with accuracy.

As shown in FIG. 2, each side of the mark M is L ₁ , L ₂ , L ₃ , L ₄ , L ₁ and L ₃ are parallel, and L ₂ and L ₄ are parallel in different directions. Now each P ₁₁ to P _1i measurement points of the points on these _{sides, P 21 ~P 2j, P 31} ~P 3k, and P ₄₁ to P _4l. Note each of the measurement points _{P 11 ~P 1i, P 21 ~P} 2j, P 31 ~P 3k, the coordinates of _{P 41 ~P 4l P 11 (X} 11, Y 11) ~P 1i (X 1 i, Y 1i) _{, P 21 (X 21, Y} 21) ~P 2j (X 1j, Y 2j), P 31 (X 31, Y 31) ~P 3k (X 3 K, Y 3k), P 41 (X 41, Y 41 ) To P _4l (X _4l , Y _4l ).

The distances between the sides L ₁ , L ₂ , L ₃ and L ₄ and the origin 0 are D ₁ , D ₂ , D ₃ and D ₄ , respectively. When the angle formed by the pair of parallel sides is φ and the inclination of the mark M is θ, the constraint condition equations shown in the equations 2 to 5 are obtained for the measurement points on each side.

  Here, if the center position of the mark M is (Xm, Ym), the following Expression 13 is obtained.

ここで、前記Ｕ，ＶはマークＭの中心位置の各平行辺の組の幅方向のずれ量であり、それぞれ下記の数１４で示される。

Here, U and V are deviation amounts in the width direction of the sets of parallel sides at the center position of the mark M, and are respectively expressed by the following Expression 14.

各平行辺の組において、平行辺上の計測点と平行辺の幅方向の位置誤差の２乗和が最小となる平行辺の位置を数１５により求める。

For each set of parallel sides, the position of the parallel side that minimizes the sum of squares of the position error in the width direction of the measurement point on the parallel side and the width of the parallel side is obtained by Equation 15.

Ｕ，Ｖの最適推定値は以下の数１６、数１７に示すように得られる。

The optimal estimated values of U and V are obtained as shown in the following equations 16 and 17.

これにより、マークＭの中心位置の最適推定値（Ｘｍ^* ，Ｙｍ^* ）は以下の数１８により得られる。

Thereby, the optimum estimated value (Xm ^* , Ym ^* ) of the center position of the mark M is obtained by the following equation (18).

これにより明らかなように、本発明においてマークの姿勢θを算出するには、前記数１１、数１２より明らかなように既知の値である平行辺の組の間の角度φと計測点の値だけで得られ、Ｄ₁ 〜Ｄ₄ を必要としない。

As is clear from this, in order to calculate the posture θ of the mark in the present invention, the angle φ between the set of parallel sides and the value of the measurement point, which are known values, are clear from the equations 11 and 12. obtained simply and does not require D ₁ to D _4.

Further, the optimum estimated value of the center position of the mark can be obtained only by φ and the value of the measurement point when θ is obtained by the above equations 11 and 12, and D _{1 to} D ₄ are not required.

  An embodiment of the present invention will be described with reference to FIGS.

  FIG. 3 is a block diagram of an embodiment of the present invention, FIG. 4 is an explanatory diagram of the operation of the present invention, FIG. 5 is an explanatory diagram of mark approximate position search by template matching, and FIG. 6 is an explanatory diagram of edge position measurement by polar coordinate development of the mark image. is there.

  In the figure, 1 is a television camera, 2 is an image input device, and 3 is an image processing computer.

  The television camera 1 captures a mark formed on a component mounting body on which a component such as a printed circuit board is mounted, and sends the captured image to the image input device 2.

  The image input device 2 digitizes the mark image input from the television camera 1 and sends it to the image processing computer 3.

  The image processing computer 3 performs image processing such as edge detection, mark detection, mark attitude calculation, mark position calculation, etc., on an image signal input from the image input device 2, and includes a display unit such as a liquid crystal display. 3-1, a main unit 3-2 having an input unit such as a calculation unit, a storage unit, and a keyboard is provided.

  The operation of the present invention will be described based on the operation state explanation flowchart shown in FIG.

  S1. A work image including a mark image obtained by photographing a work with a television camera is sent to the image input device 2 and A / D converted into a digital image. The data is sent to the main body 3-2 of the A / D converted image processing computer 3.

S2. The main body 3-2 of the image processing computer 3 searches for this image in the horizontal direction as shown in FIG. 5 in order to detect the approximate position of the mark M by template matching as in the prior art. In this manner, the image correlation between the template images TP ₁ , TP ₂ , TP ₃ ... Of the plurality of marks M recorded in advance and the input image of the mark M is performed by searching the screen. This is done by calculating the value, that is, the similarity, and obtaining the one with the highest correlation value. In this case, a template image having a plurality of pre-recorded postures, an image obtained by rotating the mark M by a prescribed angle, for example, 1 degree or 2 degrees, or an image enlarged or reduced by 1% or 2% is used. Can do. As a result, the approximate position of the mark M can be detected and the next edge point can be accurately measured.

S3. Next, as shown in FIG. 6, polar coordinate conversion is performed on the image of the edge portion of the peripheral mark M around the approximate position of the mark M. Therefore, in order to detect each side L ₁ , L ₂ , L ₃ , L ₄ of the mark M from the center point 0 of the schematic image of the mark M, a search is performed in the edge direction at every constant angle θ and The edge is measured from the change. Since the approximate position of the center point 0 of the mark M is known and the shape of the mark M is also known, when an edge search is performed from the center point 0, the sides detected by the magnitude of θ are L _{1 to} L ₄ . You can determine which one. In this way, as shown in FIG. 6, when the edge measurement is performed, the position of the edge of the mark M, the distance from the center point 0, and the angle θ can be displayed by polar coordinate conversion. And in the main-body part 3-2, what was further displayed by this polar coordinate is converted into an XY coordinate display. In this way, the following edge positions are obtained.

Ｓ４．次に、前記辺Ｌ₁ 〜Ｌ₄ について得られた前記各計測点のデータにもとづき、本体部３−２は先ず前記数１２を演算してＡ，Ｂを得る。このとき、マークＭの幅Ｗ、高さＨ、平行辺の組の間の角度φは既知であるので、これらＡ，Ｂを演算することができる。それから本体部３−２は、これらＡ，Ｂを用いて、前記数１１を演算してマークＭの姿勢θの最適推定値を得ることができる。

S4. Next, based on the data of the respective measurement points obtained for the sides L _{1 to} L ₄ , the main body 3-2 first calculates the formula 12 to obtain A and B. At this time, the width W of the mark M, the height H, and the angle φ between the pair of parallel sides are known, so these A and B can be calculated. Then, the main body section 3-2 can obtain the optimum estimated value of the posture θ of the mark M by calculating the equation 11 using these A and B.

  S5. After calculating the posture θ of the mark M in this way, the main body unit 3-2 can calculate the optimum estimated value of the center position of the mark M according to the equations 16, 17, and 18.

  S6. Then, the posture θ of the mark M and the center value (Xm, Ym) of the mark M are output.

  As described above, according to the present invention, the posture of the mark M is obtained with an accuracy less than the image resolution, and the position of the mark M is obtained with an accuracy less than the image resolution based on the obtained posture information.

  According to the present invention, the measurement accuracy of the mark position is improved as compared with the conventional sub-pixel measurement using the function approximation of the luminance change. In addition, since convergence calculation is not used, the amount of calculation is small and high-speed processing is possible. In particular, in the conventional method, it is indispensable to take a large number of measurement points on the edge in order to perform straight line fitting, and there is a problem in that the amount of processing of image measurement is large, but in the case of the present invention, each side Since the estimation calculation is performed by combining the measurement points, there is an advantage that it is possible to obtain an estimated value that requires fewer measurement points than the conventional method.

It is inclination explanatory drawing of a mark. It is position explanatory drawing of a mark. It is one Example block diagram of this invention. It is operation | movement explanatory drawing of this invention. It is explanatory drawing of a mark rough position search by template matching. It is edge position measurement explanatory drawing by polar coordinate expansion | deployment of a mark image. It is explanatory drawing of a linear equation.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Television camera 2 Image input device 3 Image processing computer 3-1 Display part 3-2 Main part

Claims (2)

An angle φ between a pair of parallel sides in the mark, a width W of the mark, and a mark height H formed on the part mounting body on which the part is placed and having two or more sets of parallel sides having different directions. A camera that shoots a mark whose shape on the screen is known,
Is input image captured by the camera, an image input unit that performs analog-digital conversion of the input data,
In the mark position / attitude detection device including an image processing computer for processing the image data output from the image input means,
In the image processing computer,
Storage means for storing image data, a detecting means for detecting the approximate position of the mark on the image data stored in the storage means, and each side of the mark from the direction from the center point of the approximate position of the mark the detected an edge measuring means for detecting the mark edge made, the coordinate values of each side detected by the edge measuring means _{(Y 1g, X 1g) (} Y 2e, X 2e) (Y 3h, X 3h) (Y 4f, X _4f ) , a mark position / posture detection apparatus comprising first calculation means for calculating the following mark posture θ with sub-pixel accuracy .

Y _1g is the Y coordinate value on the first side of the mark, Y _2e is the Y coordinate value on the second side of the mark,
Y _3h represents the Y coordinate value on the third side of the mark, Y _4f represents the Y coordinate value on the fourth side of the mark,
X _1g is the X coordinate value of Y _1g , X _2e is the X coordinate value of Y _2e ,
X _3h represents the X coordinate value of X _3h , X _4f represents the X coordinate value of Y _4f ,
φ indicates the angle between the pair of parallel sides in the mark,
W represents the width of the mark, and H represents the height of the mark. 2. The mark position / posture detection apparatus according to claim 1, further comprising second calculation means for calculating the following mark center positions Xm, Ym.

Images