JP2000180138A - Calibration plate and calibration system for visual sensor utilizing it - Google Patents

Abstract

PROBLEM TO BE SOLVED: To calibrate a visual sensor (e.g. a camera) with high accuracy. SOLUTION: Circular teaching points are arranged on a calibration table 2. First elongated reference shape and second small circle reference shape are arranged in the center thereof. Image of the calibration table 2 is taken out by means of a camera 1 and processed to extract first and second reference shapes. Origin and x-axis are detected from the first reference shape and the y-axis is detected by connecting the origin with the second reference shape. Origin and x, y axes of the calibration table 2 can be detected at high accuracy utilizing two reference shapes. Subsequently, the teaching points are detected and calibration is effected by comparing the position of teaching point in an image with the position in the actual space.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a calibration plate used for calibrating a visual sensor (for example, a camera) and a calibration apparatus using the calibration plate.

[0002]

2. Description of the Related Art In assembly robots and inspection robots, it is necessary to recognize the spatial position and shape of an object based on an image obtained by a visual sensor. For this purpose, it is necessary to preliminarily associate the image obtained by the visual sensor with the spatial position, and if the parameters such as the camera position, attitude, and focal length in the spatial coordinates are not known in advance. No.

For this purpose, it is necessary to calibrate the visual sensor.
In Japanese Patent Publication No. 40, there is proposed an apparatus using a calibration plate in which a plurality of teaching points and one reference teaching point are arranged. In this calibration plate,
An elliptical reference teaching point is arranged at one corner thereof. Then, by recognizing the major axis direction and the minor axis direction of the reference teaching point (ellipse) in the captured image, the reference point (origin) and the reference direction (x, y axes) on the calibration plate are recognized. Then, a plurality of teaching points in the image are respectively specified based on the recognized reference points and the reference directions, and the calibration of the visual sensor is performed based on the relationship between the positions on these images and the positions known in advance. Do.

As described above, by providing the reference teaching point,
Various calibration parameters can be obtained simply by photographing a predetermined calibration plate,
Calibration can be performed easily.

[0005]

However, in the device proposed above, when the calibration plate is greatly inclined with respect to the optical axis of the camera, the major axis and the minor axis of the ellipse serving as the reference teaching point can be accurately detected. There was a problem that it was difficult to do. Further, since the reference teaching points are arranged at the corners of the calibration plate, it is necessary to always include this portion in the field of view, and there is a problem that the degree of freedom in the arrangement of the camera and the calibration plate is reduced.

Japanese Patent Application Laid-Open No. 9-329418 discloses that a plurality of two types of marks (spheres), a large mark and a small mark, are arranged in a plane and a workpiece is photographed. It shows that a rough calibration parameter is obtained, the area of a small mark is estimated based on the result, the position of the small mark is accurately obtained, and calibration is performed. However, in this method, it is necessary to distinguish between a large mark and a small mark. However, depending on how the marks are placed, it is difficult to make a reliable determination because near marks appear large and distant marks appear small. There's a problem. Furthermore, it is necessary to put all the large marks in the field of view, and there is a problem that the degree of freedom of arrangement of the camera and the marks is small.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and a reference point and a reference direction can be reliably obtained.
It is another object of the present invention to provide a calibration plate for a visual sensor having a large degree of freedom in arrangement of the visual sensor and the calibration plate, and a calibration device using the same.

[0008]

SUMMARY OF THE INVENTION The present invention relates to a calibration plate used for calibrating a visual sensor, comprising: a first reference shape indicating a first reference direction according to the shape; The combination includes a second reference shape indicating a second reference direction and a plurality of teaching points, and the first reference shape, the second reference shape, and the plurality of teaching points are arranged at predetermined positions. It is characterized by the following.

As described above, in the present invention, since two reference shapes are used, the reference direction can be detected with high accuracy. It is relatively easy to determine only one direction by an elongated shape or the like. Further, a reference point is also obtained by calculating the area centroid and the like. Therefore, a reference point and a first reference direction are obtained using the first reference shape.

In the present invention, the second reference direction (for example, the y-axis) is detected with reference to the second reference shape. 2
By using the two shapes, the second reference direction is easy,
And it can be obtained with high accuracy.

It is preferable that the first reference shape is an elongated shape in which one direction is longer than a direction orthogonal to the first reference shape, and the longitudinal direction indicates the first reference direction.

That is, by increasing the aspect ratio of the first reference shape considerably, the first reference direction (for example, the x-axis) can be reliably detected even when the direction of the camera is considerably oblique by obtaining the principal axis of inertia.

The second reference shape is, for example, a circle,
It is a closed area having a characteristic that can be distinguished from other teaching points such as a square and a rhombus, and it is preferable that the second reference direction is indicated by a direction connecting the first reference shape and the center of a circle or the like.

The second reference direction can be determined by obtaining the area center of gravity of the second reference shape formed of a circle or the like and connecting this point to the reference point of the first reference shape.

It is preferable that the first and second reference shapes are arranged adjacent to each other, and the plurality of teaching points are arranged around the first and second reference shapes so as to surround the first and second reference shapes.

As described above, the first and second reference shapes are arranged adjacent to each other, and the teaching points are arranged around the first and second reference shapes. And the second reference direction are determined, and the coordinates in the image can be determined. Therefore, the degree of freedom regarding the arrangement of the camera and the calibration plate is very high.

According to the present invention, a calibration plate is photographed, the first and second reference directions are recognized based on the photographed image of the calibration plate, and the recognition is performed based on the recognized first and second reference directions. A calibration device for calibrating the visual sensor based on the image positions of a plurality of teaching points in the captured image and the positions of the teaching points known in advance on the calibration plate. It is characterized by using a plate.

[0018]

Embodiments of the present invention (hereinafter referred to as embodiments) will be described below with reference to the drawings.

FIG. 1 is a conceptual diagram of a calibration device according to the present invention. The calibration plate 2 is arranged so as to be within the field of view of a camera 1, and the calibration plate 2 is imaged by the camera 1. The obtained image is converted into a predetermined image signal by the image processing device 3 and sent to the arithmetic unit 4. The calculation unit 4 detects a reference position and a reference direction based on the mark on the calibration plate. On the other hand, from the data input unit 5, array information of marks (first and second reference shapes and a plurality of teaching points) on the calibration plate is supplied to the calculation unit. Therefore, the arithmetic unit associates the coordinate values of each mark in the image with the spatial coordinate position obtained from the position of each mark on the calibration plate from these data. Then, the calculation unit 4 performs a calibration calculation from the set of the image of each mark and the spatial coordinate values, and performs various calibration parameters (camera parameters).
Ask for. The obtained calibration parameters are
The data is sent from the data output unit 6 to another device and used for subsequent processing (for example, control of a robot hand).

FIG. 2 shows the configuration of the calibration plate 2 in the embodiment. Thus, the calibration plate 2 has the first reference shape 10, the second reference shape 12, and the plurality of teaching points 14.

The first reference shape 10 has an elongated slot shape in which both ends are semicircular. Therefore, the first reference direction can be easily detected by detecting the principal axis of inertia (longitudinal direction) of this shape. Further, the reference point (for example, the origin) can be determined by detecting the center point of the area of the first reference shape. For example, if the reference point is (100, 100,
0) may define another location as the origin (0, 0, 0).

The second reference shape 12 is a small circle, and the second reference direction can be determined by connecting the area center of gravity (center) to the reference point of the first reference shape.

As described above, in the present embodiment, since two reference shapes are used, the reference direction can be detected with high accuracy. That is, by setting the aspect ratio of the first reference shape to be considerably large (1: 2 or more, for example, 1: 3), the principal axis of inertia can be reliably detected even when the direction of the camera 1 is considerably oblique. It is also easy to determine the reference point from the area center of gravity.

On the other hand, it is very difficult to obtain a second reference direction orthogonal to the first reference direction at a reference point from one shape. In the present embodiment, the area reference center of the second reference shape composed of small circles is determined, and this is connected to the reference point to determine the second reference direction. Therefore, the detection is easy and can be performed with high accuracy.

The first and second reference shapes 10, 12
Are arranged adjacent to each other. Therefore, if these two reference shapes are within the field of view of the camera 1, the reference point and the first
And the second reference direction are determined, and the coordinates in the image can be determined. Therefore, the degree of freedom regarding the arrangement of the camera 1 and the calibration plate is very high.

Further, the first and second reference shapes 10, 12
Was arranged at the center of the calibration plate 2, and a plurality of teaching points 14 were arranged around the center. by this,
As shown in FIG. 3, the coordinate values of the teaching point can be determined regardless of where the first and second reference shapes are located in the image. In particular, according to this method, as shown in FIG. 3, the teaching points can be arranged over the entire captured image regardless of the position of the first and second reference shapes in the image. It greatly contributes to improving accuracy.

Note that the calibration plate 2
The reference shape and the teaching points may be of any material such as those printed on a plate material as long as they can be distinguished in the image. It is preferable to use one with a hole.

Next, the operation of the calibration will be described with reference to FIG. First, an image of the calibration plate 2 taken by the camera 1 is taken into the image processing device 3 (S0). This is the halftone image shown in FIG. This image includes first and second reference shapes 10,
12 is included. The image processing unit binarizes the image by comparing it with a predetermined threshold (S1). Thereby, as shown in FIG. 6, the first and second reference shapes 1
Regions of 0, 12, and the teaching point 14 and other portions are identified.

When the binarization processing is completed, the data is sent to the arithmetic unit 4 for further processing. The image processing device 3 and the arithmetic unit 4 can be constituted by one computer. The calculation unit 4 extracts the reference shape and the area of the teaching point from the binarized image, and extracts the first reference shape 1
0 is detected (S2). The detection of the first reference shape 10 is performed by finding a region having the largest length or width. That is, in the present embodiment, as shown in FIG.
About three times as large as Therefore, if the vertical and horizontal directions on the screen are detected for the extracted area, the first reference shape becomes the largest.

Next, as shown in FIG. 7, the x-axis direction (first reference direction) and the origin (reference point) are detected from the detected first reference shape 10 (S3). Here, the x-axis direction is determined by obtaining the principal axis of inertia of the extracted first reference shape 10 by a known algorithm. The origin is determined by obtaining the area centroid of the region.

Next, the y-axis direction (second reference direction) is detected (S4). To detect this y-axis direction, first,
The reference shape 12 is detected. In this embodiment, the second
The reference shape 12 is a circle smaller than the teaching point 14 and has a first shape.
It is arranged adjacent to the reference shape. Therefore, the second reference shape 12 is detected by extracting the smallest area near the origin. Then, the area center of gravity of the second reference shape 12 is detected, and the y axis is detected by connecting the center of gravity to the origin. This detection is shown in FIG.

In this way, the origin, x-axis, and y-axis of the calibration plate 2 in the pixel are specified. Therefore, the teaching points 14 in the image are searched, and labeling is performed to specify each of them, as shown in FIG. 9 (S5). Also, coordinate values on the image for each teaching point,
And spatial coordinate values. Note that the spatial coordinate value of each teaching point is known from the relative positional relationship from the origin (reference point) on the calibration plate 2.
The coordinate value of the teaching point is determined based on the area center of gravity of the region.

As a result, a list showing the relationship between coordinate values and spatial coordinate values for each teaching point is obtained as shown in Table 1.

[0034]

[Table 1] Then, calibration is executed based on the obtained correspondence, and various calibration parameters are calculated (S6). This calculation is based on Japanese Unexamined Patent Publication No.
The method described in Japanese Patent Publication No. 0 can be used as it is.

Here, the search for the teaching point in S5 will be further described with reference to FIG. First, the teaching points are sequentially searched from the x-axis origin and registered in a list. This corresponds to the corresponding point of the teaching point that exists in the positive direction of
(2,0), (3,0), (4,0) ..., (-2,0), (-3,0), (-4,0) ... in the negative direction By performing labeling, each corresponding point is made to correspond to a teaching point.

The teaching points (1, 0), (-1, 0) and (0, 1), (0, -1) do not exist in the present embodiment because the reference shapes 10, 12 exist. . The teaching point (0, 0) is replaced with the first reference point.

Next, the teaching point (n, 0) on the x-axis ｛n =＝
.., -3, -2, 0, 2, 3, 4,..., In the y-axis direction, teaching points (n, 1), (n, 2),.
-1), (n, -2),... And the corresponding points are registered in a list.

Here, the teaching point where x = 1, -1 is
Since the teaching point of (1,0) (-1,0) does not exist, the y-axis direction from the middle point of the teaching point of the origin and (2,0) or the teaching point of (-2,0) And the corresponding point is registered in the list.

Next, the teaching point (0, m) on the y-axis ｛m = ·
.., -3, -2, 2, 3, 4,... In the x-axis direction from the teaching point (1, m), (2, m),.
m), (−2, m),... and the corresponding points are registered in a list. However, the teaching points already registered in the list are not newly registered.

The teaching point where y = 1, -1 is
Since there are no teaching points of (0, -1) and (0, 1),
From the origin and the midpoint of (0,2) or (0, -2), x
Search in the axial direction and register the corresponding points in the list.

Thus, each teaching point on the image is detected.

Here, if there is distortion in the image, the x and y axis directions are curved. In this case, when the teaching point is obtained by applying the direction vectors of the x and y axes on the origin to the entire image, the larger the distance from the origin, the larger the value.
Therefore, the teaching points are searched by sequentially changing the directions of the x and y axes.

For example, when searching for the next teaching point after searching for p0 and p1 in FIG. 10, the search is performed in the direction of a straight line passing through the center between p0 and p1. p3 after detection of p2
Is searched in the direction of a straight line passing through p1 and p2. In this way, by using the immediately preceding vector for searching for the next teaching point, a correct teaching point can be searched for even if the image has a curvature.

[0044]

As described above, according to the present invention,
By employing a calibration plate having two types of reference shapes, a reference point (origin), and first and second reference directions (x, y axes) can be detected with high accuracy from an image obtained by capturing the calibration plate. it can. Therefore, the calibration of the visual sensor can be performed with high accuracy using the result. In addition, by arranging the reference shape at the center of the calibration plate, the degree of freedom of arrangement between the visual sensor and the calibration plate can be increased.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of a calibration device.

FIG. 2 is a diagram showing a configuration of a calibration plate.

FIG. 3 is a diagram illustrating the position of a basic shape.

FIG. 4 is a flowchart showing an operation.

FIG. 5 is a photograph of a halftone image showing a display example of a captured image.

FIG. 6 is a diagram illustrating detection of a binarized image and a first basic shape.

FIG. 7 is a diagram illustrating detection of an origin and an x-axis.

FIG. 8 is a diagram illustrating a second reference shape and y-axis detection.

FIG. 9 is a diagram illustrating detection of a teaching point.

FIG. 10 is a diagram showing a search for a teaching point.

[Explanation of symbols]

1 camera, 2 calibration plate, 3 image processing device, 4 operation unit, 5 data input unit, 6 data output unit, 10 first reference shape, 12 second reference shape,
14 Teaching point.

Claims (5)

[Claims] 1. A calibration plate used for calibration of a visual sensor, wherein a second reference direction is determined by a combination of a first reference shape indicating a first reference direction according to the shape thereof, and a first reference shape. And a plurality of teaching points, wherein the first reference shape, the second reference shape and the plurality of teaching points are arranged at predetermined positions. . 2. The calibration plate according to claim 1, wherein the first reference shape is an elongated shape in which one direction is longer than a direction orthogonal to the first reference shape, and the first reference direction is indicated by the longitudinal direction. A calibration plate, characterized in that: 3. The calibration plate according to claim 2, wherein the second reference shape is a closed area such as a circle, an ellipse, a rectangle, or a polygon having a reference point such as a center or a center of gravity. A calibration plate, wherein a second reference direction is indicated by a direction connecting a point serving as a reference of the first reference shape and the second reference shape. 4. The calibration plate according to claim 1, wherein said first and second reference shapes are arranged adjacent to each other, and said plurality of teaching points are first and second teaching points. A calibration plate, which surrounds a reference shape and is arranged around the reference shape. 5. Photographing a calibration plate,
The first and second reference directions are recognized based on the captured image of the calibration plate, and the recognized first and second reference directions are recognized.
A calibration device for calibrating a visual sensor based on the image positions of a plurality of teaching points in a captured image recognized based on a reference direction and the position of each teaching point known in advance on a calibration plate. A calibration device for a visual sensor, wherein the calibration plate according to any one of claims 1 to 4 is used.