JPH0661105B2 - Straight line calculation method - Google Patents

Description

The present invention relates to a position correction method using a combination of partial images in image processing, and particularly to a straight line calculation method suitable for position correction of a visual robot and the like. Regarding

[Conventional technology]

The conventional device extracts a line segment sequence from a binary image as described in JP-A-59-154578. However, no consideration was given to the movement of the line segment sequence to be extracted due to the change in the overall brightness due to the illumination or the like.

[Problems to be solved by the invention]

In the above-mentioned conventional technique, in order to recognize the linear partial shape of the recognition target, in the window defining the processing area provided in the image, the linear partial shape of the recognition target is entered,
The recognition target and the image are positioned, the partial image in the window is binarized by a fixed threshold, and the contour of the linear partial shape of the recognition target is divided into a white region and a black region. The outline of the linear partial shape of the recognition target is extracted as the boundary between the black area and the black area, and a series of pixel rows located at the boundary are linearly approximated by a constant approximate width, and the approximated line segment vertex position is set. No consideration has been given to the calculation of a straight line that represents only a certain pixel by its coordinate value, and the overall brightness changes due to changes in lighting and the like, and there are problems such as variation in quantization when binarizing with a fixed threshold value. there were.

An object of the present invention is to prevent a variation due to binarization with a fixed threshold value due to a change in overall brightness due to illumination or the like, prevent a quantization error in a linear approximation from a digital image,
It is to provide a high-speed and highly accurate straight line calculation method.

[Means for solving problems]

In the straight line calculating method in the image processing of a two-dimensional image having a straight line portion, a window is set so as to capture only the characteristic portion of the two-dimensional image, and a range including the straight line portion of the characteristic portion from the window is set. Extract the image,
The extracted characteristic portion is line-segmented to approximate a polygon, and the difference in brightness between adjacent pixels on one scanning line including the vertex of the polygon approximated is taken to maximize the difference in brightness. Point is obtained by weighted averaging weighted by the difference in brightness, and the point extracted for each vertex of the polygon approximated to the polygon is a straight line that minimizes the error from each point by the least squares method. The optimum straight line can be approximated by obtaining it.

[Action]

In a multi-valued image, by finding the point where the change in brightness on one scanning line is the maximum, the point where the change in brightness is the maximum does not change even if the overall brightness changes due to changes such as lighting. , It is possible to always extract points at the same position. Further, a straight line that minimizes the error from the obtained points can be statistically obtained by the least squares method, and can be approximated to an optimum straight line.

[Embodiment] An embodiment of the present invention will be described below with reference to FIG.

First, an automatic robot assembly system to which the present invention is applicable will be described. FIG. 2 shows a system configuration as an example. As shown in the figure, the chassis 2 that randomly flows on the belt conveyor 1 is captured by the TV camera 3, and an image of the entire chassis 2 or a partial image thereof is transferred to the image processing device 4. The image processing device 4 calculates the position and orientation of the chassis 2 from the input image and transfers the calculated position and orientation to the assembly robot 5. The assembly robot 5 assembles the parts taken out from the parts supply magazine 6 into the chassis 2,
At this time, the position and orientation data as a reference is corrected based on the information from the image processing device 4, and the parts are assembled to the upper chassis 2. Generally, the chassis 2 is in a two-dimensional arbitrary position and posture on the belt conveyor 1, but in order for the assembly robot 5 to assemble the component chassis 2 correctly, the shift amount of the chassis 2 is calculated from the input image.
With this, the reference position / orientation data may be corrected.

FIG. 3 shows various examples of the characteristic shape portion. As shown in the figure, in the image 7 obtained from the TV camera 3, a part of the substrate attached to the chassis 2 is detected as an image 8, and in the image 8, the connector image 9, the hole image 10 and the substrate edge are also detected. There are images 11 and 12, etc., which are to be extracted and processed as characteristic shape parts. In this case, if the chassis is at random, even if the position and orientation are not random,
When the windows 13 to 16 are properly set in advance, they can be extracted as those characteristic shape portions. Therefore, the extracted feature shape portion is divided into a whole feature such as the hole image 10 and a connector image 9 obtained as a part of the whole feature such as the board edges 11 and 12.

FIGS. 4 (a) and 4 (b) show examples of the extracted characteristic shape portions, respectively. Of these, the one shown in FIG. 4 (a) is a hole image 10, that is, an example in which a polygon is approximated by the line segmentation method of the entire feature, and the coordinates of the point (circle) column are stored as data. There is. In addition, as shown in FIG. 4 (b), the board edge image 11, that is, a part of the feature and the window frame 15 are polygonally approximated by the line segmentation method, and another window is further provided inside the window 15 frame. This is an example in which only the characteristic portion to be extracted is approximated to a line segment sequence by setting 18, and the coordinates of the point (circle mark) sequence are saved as data.

FIGS. 5 and 6 show a method for extracting a feature amount from the line-differentiated figure obtained in FIGS. 4 (a) and 4 (b). The center of gravity (centroid) G is required for the one that can extract the entire characteristic shape as shown in FIG. 4 (a).
The x coordinate xg and the y coordinate yg of the center of gravity G are the trapezoid Tx shown in FIG.
P ₁ , P ₂ , TyP ₁ , P ₂ as a unit, and the point sequence P is P =
{P ₁ (x ₁ , y ₁ ), P ₂ (x ₂ , y ₂ ), ..., P ₇ (x ₇ ,
y ₇ )} (where P ₈ (x ₈ , y ₈ ) = P ₁ (x ₁ , y ₁ )) is calculated from the first moments Mx, My of the entire polygon and the area S as follows. It is designed to be used.

However, Mx and My are for the x-axis and the y-axis, respectively.

In addition, according to the process shown in FIG. 1, for a part of the characteristic shape as shown in FIG. 6 which is a line segment sequence excluding the window 18 frame part in FIG. 4 (b), Approximate to the best straight line.

First, point sequence extraction (S1) is performed as shown in FIG. N
(S2) is counting the number of points. Next, as shown in FIG. 7A, the brightness of the scanning line 20 including the dot sequence on the multi-valued image 19 is changed (S3). The results are shown in FIG. 7 (b).
f is the brightness, and xi is the x coordinate on the screen. Change in brightness
Looking at 21, it can be seen that the brightness changes most near the feature value. Therefore, in order to obtain the point 22 where the brightness changes most, the difference in brightness with the adjacent pixel is calculated (S ₄ ). The results are shown in FIG. 7 (c). fxi−fxi− ₁ is the difference in brightness from the adjacent pixel, and xi is the x coordinate on the screen. In this way, it is considered that the difference in brightness near the feature value forms a nearly normal distribution, and the weighted average weighted by the difference in brightness shown below indicates that the change in brightness shown in FIG. 7 (c) is the maximum. Point 2
Extract 3. (S5).

Weighted average Performed for all the points the processing of S3~S5 more _(S 6).
FIG. 8 shows points P1 to P4 at which the change in brightness on the scanning line including each point sequence obtained by the processing of S3 to S6 is maximum. It is approximated to the straight line that minimizes the error from P _{1 to} P ₄ obtained here (S ₈ ).

As an approximation method, a straight line that minimizes the errors l _{1 to} l ₄ with P _{1 to} P ₄ shown in FIG. 8 is obtained by the least squares method shown below.

The straight line to be obtained is y = ax + b (or x = a ″ y + b ″). In this case, the coefficients a and b are the point sequences P1 to P4, and P1 = (x ₁ ,
y ₁ ), P2 = (x ₂ , y ₂ ), P3 = (x ₃ , y ₃ ), P4 =
As (x ₄ , y ₄ ), However, And Are the inverse matrix and the transposed matrix of x, respectively.

Next, a method of determining the position and orientation based on the feature amount obtained as described above will be described with reference to FIGS. 9A to 9C. Since the obtained feature amount is a center of gravity and a straight line, These combinations may be a center of gravity and a center of gravity, a center of gravity and a straight line, and a straight line and a straight line. The method of determining the position and orientation will be described below for each combination. Generally, the deviation from the reference of the position and orientation is the x direction, the y direction,
In addition to the deviation of the rotation, there is the deviation of the rotation on the xy plane.

Case: The center of gravity G1 (x ₁
g, x ₁ g), the position of the characteristic shape part, and
The displacement in the x and y directions is obtained, and the displacement angle θ in the rotation direction is obtained from G1 (x ₁ y, x ₁ g) and G2 (x ₂ g, y ₂ g) as follows. .

θ = tan ⁻¹ (y ₁ −y ₂ / x ₁ −x ₂ ) ... (8) However, G2 indicates the center of gravity corresponding to another feature shape part.

Case: Center of gravity G3 (x ₃ g, y ₃ g) as in
The positional shift can be obtained more, but next, the center of gravity G3 and the center of gravity G3
Intersection J1 with the heavy line drawn from the above to a straight line (y = ax + b)
Based on (x ′, y ′), the shift angle θ in the rotation direction is obtained as follows.

θ = tan ⁻¹ (y′−y ₃ g / x′−x ₃ g) (9): straight line (x = a ″ y + b ″) and straight line (y = ax +
The position is obtained from the intersection point J ₂ (x ″, y ″) with b), and the shift angle θ in the rotation direction is obtained from the inclination a (or a ″) of the straight line.

θ = tan ^-1 a or tan ^-1 a ″ (10) FIG. 10 shows a flow of an example of the whole position / orientation determination processing by the method shown in FIGS. 9 (a) to 9 (c). According to this, the process is explained by setting a window for the target image input from the TV camera so that an appropriate characteristic shape portion can be extracted (the processing range is determined). After that, the contours of the features in the window are extracted and then polygonal approximation is performed by the line segmentation method. After that, the feature shapes are further grasped from the line segmentation data and extracted for each feature amount type that can be extracted. Classification is performed (S13, 14). A feature amount such as a designated center of gravity or a linear equation is extracted for each of the classified feature shapes, and the posture is determined by a combination of the obtained two feature amounts. The judgment is made (S15, 16).

〔The invention's effect〕

Conventionally, when a straight line that is a characteristic shape is extracted from a binary image, when the brightness of the entire screen changes due to a change in lighting or the like, the straight line is extracted with a slight shift, not at the point where the brightness changes most. . According to the present invention, it is possible to use a multi-valued image, calculate the point where the brightness changes the most by weighted averaging, and further use the least squares method to approximate a straight line with the smallest error from each point.

[Brief description of drawings]

FIG. 1 is a processing flowchart showing one embodiment of a straight line calculating method according to the present invention, FIG. 2 is a diagram showing an apparatus configuration according to one embodiment of the present invention, and FIG. 3 is a window setting for a characteristic shape portion according to the present invention. FIGS. 4 (a) and 4 (b) show the method as a polygonal approximation of the feature shape portion extracted based on the window, and FIGS. 5 and 6 relate to the present invention. FIG. 7A is a diagram for explaining a feature amount extraction method, FIG. 7A is an example of scanning lines of a characteristic shape portion of a multivalued image, and FIG. 7B is a graph of change in brightness on one scanning line of the characteristic shape portion. FIG. 8C is a diagram showing a graph of the difference in brightness between the neighboring pixels of the characteristic shape part and FIG.
The figure is a figure for explaining the linear approximation method of the characteristic shape part,
9 (a) to 9 (c) are diagrams for schematically explaining the position / orientation determination method according to the present invention, and FIG. 10 is a diagram showing a flow of an example of the position / orientation determination process according to the present invention. 1 ... Belt conveyor, 2 ... Chassis, 3 ... TV camera, 4 ... Image processing device, 5 ... Assembly robot, 6 ...
… Magazine for parts supply.

Claims (1)

[Claims] 1. In order to recognize a linear partial shape of a recognition target, the recognition target and the image are arranged so that the linear partial shape of the recognition target is included in a window defining a processing area provided in the image. Is positioned, and the partial image in the window is binarized by a fixed threshold value so as to be divided into a white region and a black region according to the contour of the linear partial shape of the recognition target. The contour of the linear partial shape of the recognition target is extracted as a boundary of, and a series of pixel rows located at the boundary are linearly approximated by a constant approximation width,
In a straight line calculation method in which only the pixels at the apex positions of the approximated line segments are represented by their coordinate values, the pixels at the apex positions of the line segments obtained by linearly approximating the contour of the linear partial shape of the recognition target are obtained. For each pixel, the following processing is performed on the multi-valued image, the brightness of a series of pixels on the scanning line passing through the pixel is extracted, and the brightness of the adjacent pixels of the series of pixels on the scanning line is calculated. The difference is obtained, and the coordinate value of the series of pixels on the scanning line in the scanning line direction is weighted with the difference from the brightness of the adjacent pixel, and the weighted average is taken in the scanning line direction to find the maximum change in brightness. Then, a point on the scanning line that maximizes the change in brightness on the scanning line passing through each pixel at the vertex position of the line segment is obtained by repeating the above series of processing, and the brightness on each scanning line is calculated. Is the maximum change in Linear calculation method, wherein a line error is minimized, is calculated using the least squares method.