JP2505941B2 - Parts status detection method - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention detects the state (position, posture, etc.) of a component installed on a circuit board when mounting a component such as a circuit element at a predetermined position on the circuit board. On how to do.

[0002]

2. Description of the Related Art Conventionally, as a method useful for detecting the state of an object such as the above-mentioned component, the contour (edge) of the object is used.
There is known an image processing technique for extracting the. In this image processing method, the edge is a place where the brightness changes abruptly, so the differential value of the change in the brightness of the image is obtained, and this is used to extract the edge element from the image of the object. It is based on that. The differentiation here is x and y
This is a spatial first-order differentiation (gradient) for differentiating the grayscale change of an image according to coordinates, and the magnitude of the gradient is an amount corresponding to the edge strength.

If this method is applied to the above-described component mounting, the following points become problems. (1) If the contrast between the board and the component is weak, edge extraction becomes impossible.

(2) It is necessary to determine a threshold value of a differential value for each combination of a component and a board or a mounting position on the board.

(3) In the case of parts colored in a plurality of colors such as electric resistance and other circuit elements, only one edge can be extracted with one threshold value.

(4) When the state (position, posture) of the component changes, the relative relationship with the substrate also changes, so that the differential value of the edge changes, and the threshold value set in advance becomes inappropriate.

Therefore, the edge 3 of the component on the circuit board is
It has been proposed to extract an edge at a point (position) using not only the differential value but also a profile of the differential value (shape when the differential value is plotted on a graph).

However, according to this method, the following points become problems. (1) It is necessary to set profiles for all combinations of parts and positions on the board.

(2) When the state of the part changes, the profile of the point to be detected becomes different from the profile of the same type part registered in advance.

(3) Since the profiles in the horizontal direction and the vertical direction are registered, even if the density of the board portion is uniform and the situation of the above (2) is not achieved, the profile changes when the component tilts.

As described above, when detecting the position and orientation of a component from the contour image of the component mounted on the circuit board, it is difficult to stably obtain the contour of the component, and the edge of the circuit board is also included. Therefore, an image with a lot of noise is targeted.

Analytical figures (figures whose shapes can be described by equations including parameters) from such noisy images
The Hough transform is known as a typical method for detecting the noise (US Pat. No. 3,069,654). Applying this to rectangle detection results in:

As shown in FIG. 15, the size (known) of a rectangle on the xy coordinate plane is represented by p _xl (long side length) and p _yl.
(The length of the short side), the inclination of the rectangle with respect to the x coordinate axis is Θ,
The center coordinates of the rectangle are (X, Y), and the four sides A of the rectangle are
The range of the parameters s and t indicating the position on each side of the point (x, y) on B, C and D is determined as follows.

A: 0 ≤ s ≤ p _xl , t = 0 B: 0 ≤ s ≤ p _xl , t = _pyl C: s = 0, 0 ≤ t ≤ p _yl D: s = p _xl , 0 ≤ t ≤ _pyl At this time, the point (x, y) on the rectangle is represented by the following equation.

[0015]

[Equation 1]

In the above equation, the coordinates (x,
y) is substituted, parameters (s, t, Θ) are changed to calculate (X, Y) corresponding to the respective values, and the obtained point (X, Y
Y, Θ) is voted in the three-dimensional space of X, Y, Θ. As a result, it is considered that the point (X, Y, Θ) with the highest degree of integration in the three-dimensional space shows the center and inclination of the rectangle.

[0017]

However, if this method is directly applied to the detection of the component position on the circuit board,
False recognition may occur. This is because in the image of the circuit board and the mounted components as shown in FIG. 3, when the solder attachment portion 3 for electrically connecting the terminal portion of the component 2 to the conductive portion on the circuit board 1 is deformed due to the soldered shape. The number of voting values from the contour portion (pixel) to the three-dimensional space decreases, and there is a rectangular pattern formed by the conductive portion on the circuit board or the edge inside the component contour. This is because the integrated value of the rectangular pattern other than the component also becomes large, and the center position and inclination of the component deviate from their correct values.

Therefore, an object of the present invention is to provide a method capable of detecting the position, posture, etc. of a component on a circuit board without erroneous recognition by utilizing the Hough transform.

[0019]

According to the present invention, the gradient direction is unstable toward the inside of the component on the side where the solder is joined among the sides showing the contour of the component in the image as shown in FIG. On the other hand, in view of the property that the solder is stable on the side where the solder does not join, when applying the above-mentioned Hough transform to the state detection of the part, the characteristics of the contour of the part are used (for example, on the side where the solder does not join, Voted with a weight, to make a difference with the side where the solder is joined).

As a configuration for that purpose, the present invention is a method for detecting the position and orientation of a component installed on a circuit board, and the gradient of each pixel obtained by differentiating the brightness of an image including the image of the component. Based on the extracted contour of the part in the image, the direction of the gradient to determine which side each pixel is on the contour of the part,
The state of the component is detected by weighting each pixel according to the result and performing Hough transform.

As an additional configuration of the present invention, the component inclination angle is detected from the distribution of the pixels forming the image in the gradient direction, and the detected angle and a predetermined angle difference from the angle are detected. The Hough transform is repeatedly executed with the two angles as the inclination angle of the component until the integrated value decreases, and when the integrated value of the Hough transform for the latter two angles decreases, the component detection position immediately before that and The tilt angle may be the position and tilt of the component. Further, in this case, in the second and subsequent Hough transforms, it is preferable to limit the parameter plane for voting and to calculate in advance the range of pixels and parameters for voting in the area on the limited plane.

[0022]

According to the present invention, the magnitude and direction of the gradient of the pixels forming the image are obtained by differentiating the brightness of the image including the image of the part obtained by the image pickup device such as the CCD camera. While the size of the gradient is used to extract the outline of the part in the image, the direction of the gradient is used to determine which side of the outline of the part each pixel is. Then, by weighting each pixel based on the determination result and performing the Hough transform as described above, the position and orientation of the component can be detected more accurately.

Further, the component inclination angle is detected from the gradient direction distribution of the pixels forming the image, and the Hough transform is repeatedly executed until the integrated value decreases to detect the position and inclination of the component. The detection accuracy is improved.

Further, in the second and subsequent Hough transforms, by pre-calculating the range of pixels and parameters to vote in the limited area on the plane, the overall calculation amount is reduced and the detection speed is increased.

[0025]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT FIG. 1 shows a system configuration for implementing the method of the present invention. This system is provided with a CCD camera 11 as an image pickup device for obtaining a grayscale image as shown in FIG. 3 including a component 2 mounted on a circuit board 1, and an image signal to be sent from the camera 11 will be described later. A personal computer 12 is used as processing means for performing processing operations.
And a monitor C for displaying its processing operation and image data
And RT13.

FIG. 2 is a flow chart showing the processing procedure of the method of the present invention executed by the personal computer 12. The outline of this procedure is as follows.

First, in the grayscale image obtained by the CCD camera 11, the contour of the component is extracted,
As shown in FIG. 4, an image with an enhanced contour is obtained. At that time, the pixels on the contour of the component are weighted based on the "continuity of edges" described later (step ST1). next,
The component inclination angle is detected from the gradient direction distribution of the pixels forming the image (step ST2), and each pixel is weighted by the detected inclination angle (step ST3). As a result, an image in which the contour of the component is weighted (FIG. 14) and the inclination angle of the component are obtained.
On the other hand, a rectangular Hough transform with a fixed inclination angle is performed to detect the center position of the component (step ST4). Each step will be described below.

First, the principle of component contour extraction will be described.

First, the magnitude and direction of the gradient obtained by differentiating the light and shade (brightness) of the image are obtained. Assuming that the brightness of the image is f (x, y) on the xy plane shown in FIG. 5, the gradient ∇f is represented by the sum of the differential value vector dx in the x direction and the differential value vector dy in the y direction. The gradient magnitude (edge strength) is expressed by the following equation.

[0030]

[Equation 2]

Further, the gradient direction is the angle θ of the vector from the dark to the light, which is perpendicular to the boundary line (the contour of the detection target) between the light and the dark of the image.

In the image of FIG. 3 again, the edges of the component 2 are straight or gently curved, while the edges of the substrate 1 (ground portion of the image) are randomly distributed. The contour of the part is extracted using the idea of "sex". This "edge continuity" refers to the edge point of interest (the point where the edge strength has a value equal to or greater than a threshold value).
It means that the direction of the edge of is the same as the direction of the edge of the edge point adjacent thereto. This idea of “continuity of edges” is a concept known in the art of image recognition.

The method of determining the threshold value for obtaining the above-mentioned edge point is as follows. That is, the edge strength at the point (i, j) in the pixels forming the image is e _ij , and the strength e _{ij
Assuming that} a predetermined ratio (R%) of the maximum value e _max of the above is a threshold T _H , a point (i, j) satisfying e _ij ≧ T _H is an edge point.

Here, the directions of the eight neighboring points (pixels) viewed from the point of interest (i, j) are represented by an integer k as shown in FIG. Further, as shown in FIG. 7, the direction of the edge of the point of interest (i, j) is d, and the direction rotated counterclockwise by π / 2 is d + π / 2. An adjacent point in this direction is referred to as a left adjacent point of the point of interest because it is located on the left side along the edge when standing at the point of interest and facing the direction of the edge.

When both the adjacent point in the direction k and the point of interest are edge points, in order to see the continuity of the edge, whether or not both points are lined up along the edge and the direction of the edge are Whether or not they are complete must be evaluated.

Now, it is checked whether the edge point in the direction k is the left adjacent point to the point of interest and whether the edge directions are aligned. The evaluation function on the left side is L (k), and the evaluation function on the right side is R as well. Assuming (k), each evaluation function is given by the following equation.

[0037]

(Equation 3)

Here, the function a is

[0039]

[Equation 4]

Where | α-β | is the absolute value of the difference between the angles α and β.

A (α, β) takes a larger value as the values of α and β are closer to each other. When α = β, the maximum value is a (α, β) = 1,
When α and β are opposite, the minimum is a (α, β) = 0. That is, 0 ≦ a (α, β) ≦ 1.

Therefore, a (d, d _k ) in the equation (3) is a factor for evaluating how well the edges of the point of interest and the adjacent points of the direction k are aligned, and a (πk / 4, d + π / 2) is a factor for evaluating whether or not the adjacent point in the direction k is the left adjacent point. For example, when the edge direction of the point of interest is d = 0, the left adjacent point is k = 2, but at this time, d + π / 2 = πk / 4, and thus a (πk / 4, d + π / 2) = 0. . Further, if d = d _k , a (d, d _k ) = 1, and therefore L (2) = 1, which is the maximum value. The same applies to the right side. As described above, the higher the continuity of the edge, the larger the values of L (k) and R (k).
These are called continuity evaluation functions. Since the edge continuity of each pixel is evaluated by using such an evaluation function, the pixels on the contour of the component can be weighted based on the evaluation.

The contour of the component may be extracted by a method other than the edge continuity evaluation.

Returning to FIG. 2, after weighting the pixels on the component outline in step ST1, the inclination (inclination angle) of the component is detected (step ST2). This is done as follows.

First, the gradient direction (angle θ) is obtained for all pixels of the image and plotted on a plane with the horizontal axis representing the angle and the vertical axis representing the number of pixels. For example, if the angle is
At the 90 ° position, the number of pixels with a gradient of 90 ° is plotted. As a result, a histogram in the range of ± 180 ° as shown in FIG. 8 is obtained.

Since the target image contains many patterns having many horizontal and vertical components indicating the contour of the part, the above histogram has a distribution of about 90 ° period. Therefore, by superimposing the histograms at the turning points of ± 45 ° and ± 135 °, a histogram with an angle range of ± 45 ° as shown in FIG. 9 is created. This makes 90
The differential direction of the parts distributed every ° becomes clearer.

Next, a Gaussian function filter as shown in FIG. 10 is applied (filtering) to create a histogram in which the peak (maximum value) becomes clear as shown in FIG. Then, the above filtering is repeated until the number of local maximums above the noise level in this histogram becomes 2 or less. When the number of maximum values is 2 or less, if there is one maximum value, the angle that takes the maximum value is the inclination of the component, and if there are two maximum values, the angle with the larger absolute value is the component angle. Slope In the example shown in FIG.
Since the peak near 0 ° is the horizontal or vertical component of the pattern, the angle of 16 ° having the larger absolute value is the inclination of the component.

In step ST2 of FIG. 2, the component inclination angle is detected from the distribution of the gradient directions of all the images by the above procedure, so that the component position can be corrected based on the detection result.

Next, in step ST3, each pixel is weighted by the tilt angle detected as described above. For example, a new weight is obtained by multiplying the weight w of each pixel by a weight coefficient f as shown in FIG. That is, when the detected component inclination angle is Θ, the gradient direction is θ, and the absolute value | Θ−θ | of the difference between them is Δθ, new weight = w · f (Δθ) (6) The weight coefficient f is When Δθ <a, f = 1 When a ≦ Δθ ≦ b f = (b−Δθ) / (b−a) When b <Δθ, f = 0. As a result, an image in which pixels are weighted according to the direction (tilt angle) of the contour of the component and the tilt angle of the component are obtained as shown in FIG.

Next, in step ST4 of FIG. 2, the inclination angle of the component is fixed and the center position of the component is detected by the Hough transform (FIG. 15) described above.

At this time, first, it is checked which side of the rectangle in FIG. 15 each pixel may be on, depending on the gradient direction of each pixel. As an example, it is assumed that the sides C and D of the rectangle in FIG. 15 are the sides connected to the solder of the mounting component and the sides A and B are the sides not connected to the solder. In the image, the solder portion appears dark, and the vicinity of the connection side with the solder inside the outline of the part appears bright because the preliminary solder has been applied. Therefore, the gradients of the pixels on the sides C and D face each other toward the inside of the contour of the component. On the other hand, side A,
The gradient of each pixel on B cannot be uniquely determined whether it is inside or outside the contour of the component due to the relative relationship between the brightness of the component and the brightness of the substrate. Therefore, it is possible to determine which side each pixel may be on from FIG. 16 from the inclination angle of the component and the gradient direction of each pixel.

Next, as shown in FIG. 17, regarding the points considered to be on the upper side A or the lower side B, the parameters s and t are determined at the points which are the centers of the parts when the upper side A and the lower side B are considered. Vote sequentially. For points that are considered to be on the left side C, vote with a plus (+) attached to the point that is the center of the part when you think of the left side C, and minus if you think of the right side D. Vote with (-). On the other hand, regarding the point that is considered to be on the right side D, the left side C
Vote with + and – opposite to the case of. Further, when these votes are cast, the points considered to be on the upper side A or the lower side B are weighted more heavily than the points considered to be on the left side C or the right side D. Note that adding + and-is equivalent to adding weights of +1 and -1.

In this way, since the + and-votes are performed at the position where the conventional Hough transform method is likely to erroneously recognize a rectangle or a part having a shape close to that, the accumulated value is canceled and the upper side and the lower side with large weights are cancelled. Since the voting value from is large, the above-mentioned erroneous recognition is eliminated.

In the above method, the inclination angle of the component is detected in step ST2 before Hough conversion. By doing so, it is not necessary to change the angle parameter Θ at the time of Hough transform, and the calculation speed can be increased. On the other hand, since the angle is detected without using the position information (center position etc.) of each pixel, an error may occur when the tilt angle is close to 0 °. Therefore, the method shown in FIG. 18 can be used in order to prevent the occurrence of such an error and enable more accurate detection.

Referring to FIG. 18, first, referring to FIG.
In step ST2, the Hough transform is performed using the inclination angle Θ detected as the initial value of the component inclination angle θ, and the center position P _{0 of the} component is calculated.
To detect. The integrated value at this time is set to m ₀ (step ST5).

Next, a proper angle step (angle difference) dθ is set, and two angles of large and small Θ−dθ based on the initial value Θ are set.
And Θ + dθ are subjected to Hough transform (steps ST6 and ST7). In each case, the component center position is P ₁ , P _-1 , and the integrated value is m ₁ , m _-1 . If the integrated value in the case of these two angles is larger than the initial value Θ, it is assumed that an angle error has occurred, and the Hough transform in which the angle is changed by dθ in the direction of the integrated value in these cases is Repeat until the accumulated value becomes smaller. As a result, when the accumulated value decreases at the nth time, (n-
1) The detected position and angle for the second time are the center position and posture of the component.

Specifically, in step ST8, it is determined whether or not the integrated values m ₀ > m ₁ and m ₀ > m ₋₁ . If YES, the process ends. At this time, the center position and orientation of the component are P ₀ and Θ, respectively.

On the other hand, if the determination in step ST8 is NO, in the next step ST9, it is determined whether or not m ₁ > m ₋₁ . As a result, if YES, the number n of Hough transforms is set to 1 (step ST10), 1 is added thereafter (step ST11), and the component inclination angle θ = Θ + dθ.
Hough transformation is performed as × n (step ST12: component center position P _{n at} this time, integrated value m _n ), and it is determined whether m _n > m _n-1 (step ST13). If NO, the process ends. At this time, the center position and orientation of the component are P _n-1 and Θ + dθ × (n-1), respectively. If YES, the process returns to step ST11 and the subsequent operations are repeated.

In step ST9, if m ₁ > m _{-1 is} not set, the number of Hough transforms n is set to -1 (step S 9
T14), after which 1 is subtracted (step ST15), Hough transformation is performed with the component inclination angle θ = Θ + dθ × n (step ST16: component center position P _{n at} this time, integrated value m _n ), and m _n > m _It is determined whether it is _n-1 (step ST
17), if NO, end. If YES, the process returns to step ST15 and the subsequent operations are repeated.

According to this method, the integrated value of the Hough transform becomes small, and the accuracy of detecting the center position and posture of the component is improved. That is, it is possible to efficiently improve the detection accuracy by using the tendency of the integrated value of the Hough transform.

However, this method has a problem that the amount of calculation is significantly increased by repeating Hough transform. Therefore, in the second and subsequent Hough transforms, the parameter plane (XY coordinate plane) for voting is restricted, and the range of pixels (x, y) and parameters s, t for voting in this area is calculated in advance, The amount of calculation can be reduced.

Specifically, when the actual tilt angle is Θ,
Assuming that the error in the detected angle is θ, the state of accumulation from the points on each side of the rectangle is as shown in FIG.
The things that should overlap should shift. Figure 1 shows the length of each side
When expressed by p _xl (long side) and p _yl (short side) of 5, the detected position coordinates (X, Y) may be displaced with p _xl sin θ and p _yl sin θ as the maximum widths. In other words, the true component center position (X, Y) exists somewhere within this range even if the error θ occurs. Therefore, when the Hough transform is repeated, when performing the nth Hough transform, the region to vote may be narrowed down to the periphery of the (n-1) th detected position as shown in FIG. By doing so, the second and subsequent conversions can be significantly speeded up.

Specifically, the following equation (7) and the above equation
The range of the parameters s and t in (1) and the equation (8) that limits the area of the XY plane are combined to form the parameters s and t of each pixel.
Find the range in advance.

[0064]

(Equation 5)

However, X _s , X _e and Y _s , Y _e are as shown in FIG.
The upper and lower limits of the rectangular area shown in 0 in the X-axis and Y-axis directions.

FIG. 21 shows a procedure for this speedup.

First, the Hough transform is performed using the inclination angle Θ detected in step ST2 of FIG. 2 as an initial value of the component inclination angle θ, and the center position P _{0 of the} component is detected. The integrated value at this time is set to m ₀ (step ST21).

Next, let P be the area for voting the center position of the part.
_It is limited to the area A ₀ centered at ₀ (step ST2).
2). Then, similarly to the method of FIG. 18, an appropriate angle step (angle difference) dθ is set, and the initial value Θ is used as a reference.
The Hough transform is executed in the restricted area A ₀ for each of the angles Θ−dθ and Θ + dθ (steps ST23 and ST24). After this, the method is the same as that of FIG. 18, and the procedure is performed from step ST8 onward.

[0069]

Since the method of the present invention is carried out in the above procedure, the position and orientation of the component on the circuit board can be detected by Hough transformation without erroneous recognition.

Further, the tendency of the integrated value of the Hough transform can be used to efficiently improve the detection accuracy, and further, the region for voting by the Hough transform can be limited to speed up the arithmetic processing.

[Brief description of drawings]

FIG. 1 is a system configuration diagram for implementing a method of the present invention.

FIG. 2 is a flowchart showing a processing procedure of the method of the present invention.

FIG. 3 is a diagram showing an example of images of a circuit board and mounted components.

4 is a diagram showing a contour image of a part or the like extracted from the image of FIG.

FIG. 5 is an explanatory diagram showing a gradient direction in a grayscale image.

FIG. 6 is a diagram showing the direction of an edge of a point of interest.

FIG. 7 is a diagram showing the direction of an adjacent point, the direction of an edge, and the direction of a left adjacent point viewed from the point of interest.

FIG. 8 is a diagram showing an example of a histogram with respect to a gradient angle.

9 is a diagram showing a histogram in a range of ± 45 ° obtained by superimposing the histogram of FIG. 5 at 90 ° intervals.

10 shows a Gaussian function for filtering the histogram of FIG.

FIG. 11 is a diagram showing a histogram obtained by filtering the histogram of FIG. 9 with a Gaussian function.

FIG. 12 is a diagram showing a histogram finally obtained by further filtering the histogram of FIG. 11 with a Gaussian function.

FIG. 13 is a diagram showing an example of weighting factors for weighting according to the direction of a component contour.

14 is a diagram showing an example of an image obtained from the image of FIG. 4 by weighting according to the direction of the component contour.

FIG. 15 is an explanatory diagram of parameters of rectangular Hough transform.

16 is a diagram showing on which side of the rectangle in FIG. 15 each pixel may be located.

FIG. 17 is an explanatory diagram of a plane for voting the weight of each pixel by Hough transform.

FIG. 18 is a flowchart showing a procedure for enabling more highly accurate detection in the method of the present invention.

FIG. 19 is a diagram showing a shift possibility region of an accumulation point due to an angle error.

FIG. 20 is a diagram showing a limited integration region for speeding up Hough transform.

FIG. 21 is a flowchart showing a procedure for speeding up by limiting the area of the accumulation points in the method of the present invention.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Circuit board, 2 ... Component, 3 ... Solder adhesion part, 11 ... C
CD camera, 12 ... PC, 13 ... Monitor CRT.

Claims (3)

(57) [Claims] 1. A method for detecting a state such as a position of a component installed on a substrate, wherein a pixel in the image is obtained based on a gradient of each pixel obtained by differentiating a brightness of an image including an image of the component. The contour of the component is extracted, which side of the contour each pixel is on is determined by the direction of the gradient, and the Hough transform is performed by weighting each pixel according to the result, thereby performing the component transformation. A method of detecting a component state, which comprises detecting the state of the component. 2. A component state detecting method according to claim 1, wherein a component inclination angle is detected from a distribution of pixels forming the image in a gradient direction, and the detected angle and a predetermined angle difference from the detected angle are detected. The Hough transform is repeatedly executed until the integrated value decreases, with the two large and small angles respectively set as the inclination angle of the component, and when the integrated value of the Hough transform in the case of the latter two angles decreases, the component detection immediately before that is detected. A component state detecting method comprising detecting a position and an inclination angle as a position and an inclination of the component. 3. The component state detection method according to claim 2, wherein in the second and subsequent Hough transforms, the parameter plane to be voted is limited, and the range of pixels and parameters to be voted in the area on the limited plane is determined in advance. A method for detecting a component state, which comprises calculating.