JPH06214622A - Work position sensor - Google Patents

Abstract

PURPOSE:To deal with various work shapes and also to process these works at a high speed and low cost by obtaining three or more coordinates which are not set on the same straight line on work as the data on the position and the posture of the work. CONSTITUTION:The TV cameras 3-5 photograph the images near the feature points P1-P3 which are not set on the same straight light on the surface of a work 1 and output these photographed images to the image processor 7-9. The processors 7-9 detect the coordinates of the points P1-P3 based on each image signal. A computing element 10 calculates the data to specify the position and the posture of the work 1 in a world coordinate system 2 based on the distance fata corresponding to the car type of the work 1 stored in a storage 11, the data on the installing conditions of the cameras 3-5, and the coordinates of the point P1-P3 supplied from the processors 7-9. Then the element 10 outputs the calculated data to a robot controller 12. Thus it is possible to deal with various work shapes and also to process these works at a high speed and low cost.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a work position detecting device suitable for use in an automatic production line device having a robot or the like.

[0002]

2. Description of the Related Art In an automatic production line, it is necessary for each production apparatus to perform an appropriate work on a work flowing in the line, and for that purpose, it is necessary to accurately detect the position and orientation of the incoming work. Conventionally, there have been the following two methods for detecting the position of such a work. <First method> Workpieces can be moved in parallel or in rotation 6
It is assumed that the object has a degree of freedom, and its position / orientation is calculated. When the work position is detected, three points that are not on the same straight line on the work, such as holes, are defined as representative points. Then, each of these representative points is photographed by a stereo camera method in which two cameras are seen from different directions, and the position of the work in the work space and Ask for posture.

<Second Method> First and second sides that are not parallel to each other are selected from the respective sides that define the outer shape of the work, and the positional relationship between these two sides is stored in advance in a memory. . Then, when the work arrives in the work space, two intersecting points where the first side of the work intersects the mutually parallel first and second detection planes set in the work space are detected by, for example, optical means. To do. Further, the intersection point where the second side of the work intersects the third detection plane that is non-parallel to the first and second detection planes is detected. Then, based on the coordinates of these three intersections, position information regarding the first and second sides is obtained, and the position of the work is detected based on these position information. The applicant of the present application has proposed a work position detecting device based on this method in Japanese Patent Application No. 2-9541.

[0004]

However, the above-mentioned first method has the following problems. In order to obtain the position of the work having 6 degrees of freedom, 6 TV cameras and 6 image processing devices are required, which makes the device for carrying out the method complicated and expensive. Calibration is required to perform conversion from one set of images to coordinates in the actual working space, and since this must be performed for three sets, adjustment requires a lot of time. Since the contrast of the representative points on the work is not always good, it was necessary to perform grayscale processing on the image, and it took a lot of time for detection.

Further, the above-mentioned second method has the following problems. It cannot be applied to a work that does not have two non-parallel sides. A complicated calculation is required to obtain the positional relationship between the two sides, which is difficult to carry out. The first to third detection planes are used as the first and second workpieces.
There is a need for a means for intersecting the sides of and causing the three intersections in a detectable manner. The above application (Japanese Patent Application No. 2-9541) discloses an embodiment in which this means is embodied by an optical means such as a line marker.

The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a work position detecting device which can cope with works of various shapes and which is low in price and has a high processing speed. .

[0007]

According to a first aspect of the present invention, there is provided a work position detecting device, a photographing means for photographing images of at least three characteristic points which are not on the same straight line on the work, and the photographing means. Detecting means for detecting coordinates of each feature point in the image plane, and calculating means for calculating coordinates of these feature points in a predetermined reference coordinate system based on the coordinates of each feature point in the image plane. It is characterized by including.

Further, in addition to the above configuration, the work position detecting apparatus according to a second aspect of the present invention further comprises storage means for storing coordinates in the reference coordinate system of each of the feature points, and a reference for each feature point of the new work. When the coordinates in the coordinate system are calculated, the positional deviation detecting means for calculating the positional deviation of the new work based on the coordinates and the coordinates stored in the storage means is provided.

[0009]

According to the first aspect of the invention, each feature point on the work is photographed, and the position of each feature point in the reference coordinate system is obtained based on the coordinates in the image plane. . By thus obtaining the coordinates of at least three points on the work, the position and posture of the work are determined, and it becomes possible to perform an appropriate work on the work.

According to the second aspect of the present invention, by storing the coordinates of the characteristic points of the work in the storage means, it is possible to detect the positional deviation of the new work with respect to the work thereafter.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

A: Structure of Embodiment FIG. 1 is an overall view showing a work position detecting apparatus according to this embodiment installed in a certain process of an automobile production line. In this figure, reference numeral 1 denotes an automobile body serving as a work, which is carried into the work space of this process by a conveyor (not shown) or the like. Reference numeral 2 is a world coordinate system fixed in the work space. This world coordinate system is an orthogonal coordinate system composed of x-axis, y-axis and z-axis, and is used as a reference coordinate for specifying the position of the work in this embodiment.

3, 4 and 5 are television cameras (hereinafter,
This is an abbreviated TV camera), and images of each of the three feature points P1, P2, and P3 in the vicinity of the surface of the work 1 that are not on the same straight line are captured. The images photographed in this way are output as photographing signals to the image processing devices 7 to 9 described later. TV
The size of the field of view of the cameras 3 to 5 is determined depending on whether the feature point is photographed in a state where the work 1 is stopped in the work space or when the work 1 is continuously passed through the work space. That is, when the work 1 is photographed in a stopped state and the stopping accuracy is relatively good, “± number 1
0 cm ". Further, when photographing the work 1 in the conveyed state, the size of the field of view is set to about “1 m”. This is because it is necessary to continuously detect about "1 m" when the work 1 is in the transporting state.

An example of images taken by the TV cameras 3 to 5 is shown in FIG. In the figure, 13 is an image near the feature point P1 taken by the TV camera 3, and (u
1, v1) are the coordinates of the feature point P1 in the image plane. Similarly, 14 is an image near the feature point P2, (u2, v2) is coordinates of the feature point P2 in the image plane, 15 is an image near the feature point P3, and (u3, v3) is a feature point in the image plane. These are the coordinates of P3.

Reference numeral 6 denotes a controller, which is composed of image processing devices 7 to 9, an arithmetic unit 10 and a storage device 11, as shown in FIG. The image processing devices 7 to 9 are TV cameras 3 to
The coordinates (u1, v1) of the characteristic point P1, the coordinates (u2, v2) of the characteristic point P2, and the coordinates (u3, v3) of the characteristic point P3 are detected based on the photographing signal supplied from 5.

Next, 11 is a storage device, which is distance data L1, j representing the distance between the feature points P1 and P2 on the work 1.
And distance data L2, which represents the distance between the feature points P2 and P3,
Distance data L representing the distance between j and the feature points P3 and P1
No. 3 and No. 1 of the car body, which is the work 1 Store every j. In addition, the above-mentioned data regarding the installation conditions of the TV cameras 3 to 5, that is, the following data are stored. Position vector r1, position vector r2, position vector r3 when the installation positions of the TV cameras 3 to 5 are viewed from the origin of the world coordinate system 2. Horizontal and vertical angles (θz1, θx1) of the shooting directions of the TV cameras 3 to 5 ), (Θz2, θx2), (θz3, θx3) In the case of the present embodiment, the TV cameras 3 to 5 do not have the same shooting direction. Information about the viewing angle of each of the TV cameras 3 to 5

The arithmetic unit 10 is a vehicle model No. of the work 1 stored in the storage device 11. distance data Lj, 1, corresponding to j,
Lj, 2 and Lj, 3, data relating to the installation conditions of the TV cameras 3 to 5, and the coordinates (ui, vi) of the characteristic points P1 to P3 supplied from the image processing devices 7 to 9 (i = 1, 2). , 3)
Based on and, data for specifying the position and orientation of the work 1 in the world coordinate system 2 is calculated and output to the robot controller 12. The robot controller 12 outputs a signal for controlling a robot (not shown) based on the data obtained from the arithmetic unit 10.

B: Operation of Embodiment Next, the operation of this embodiment will be described. <Teaching> When performing work on the work 1, it is necessary to teach the work. The operation according to this teaching will be described below. First, the work 1 is loaded into the work space, and the vehicle type No. j is read, and this vehicle type No. j
And a command for instructing the teaching of the reference position of the work are sent from the robot controller 12 to the computing unit 12. As a result, the arithmetic unit 10 executes the routine shown in the PAD diagram in FIG.

First, in step s1, the arithmetic unit 10
Is the vehicle model No. of Work 1. j and the command.
Next, in step s2, the arithmetic unit 10 sends a feature point detection command to the image processing devices 7 to 9 instructing that the coordinates of the feature points P1 to P3 should be detected. Then, while monitoring the states of the image processing devices 7 to 9 (step s4), the process waits until the detection of the coordinates of the characteristic points by the image processing devices 7 to 9 is completed (step s3). Note that the coordinates (ui, v
When the detection of i) is not completed, an error signal is sent to the robot controller 12 and the subsequent processing is stopped.

When the detection of the coordinates of the characteristic points by the image processing devices 7 to 9 is completed, the processing proceeds to step s5, and the arithmetic unit 10 causes the image processing devices 7 to 9 to coordinate (ui, v) the characteristic points P1 to P3.
i) (i = 1 to 3) is received. Next, in step s6, the installation conditions of the TV cameras 3 to 5 stored in the storage device 11, the viewing angle, and the vehicle model number. Work 1 corresponding to j
Distance data L1, j, distance data L2, j, distance data L3,
read j

Next, in step s7, the image processing device 7
9 to the coordinates (ui, vi) of the characteristic points P1 to P3 (i =
1, 2, 3), and the installation conditions and viewing angles of the TV cameras 3 to 5 read out from the storage device 11, the direction vector (unit vector) from each TV camera 3 to 5 toward the characteristic points P1 to P3. The components of e1, e2 and e3 in the world coordinate system 2, that is, the x component, y component and z component for each direction vector e1, e2 and e3 are calculated. Hereinafter, a method of calculating these direction vectors will be described.

First, the coordinates (ui, vi) (i = 1, 2,
3), based on the viewing angles of the TV cameras 3 to 5, a unit vector e1 ', a unit vector e2', and a unit vector e3 'indicating the directions of the characteristic points P1 to P3 viewed from the orthogonal coordinate system fixed to each TV camera. Is required. Here, the orthogonal coordinate system fixed to each TV camera means a coordinate system constituted by the following coordinate axes. An axis that corresponds to the shooting direction of the camera (direction in which the center of the field of view is viewed from the camera). All points on this axis are imaged in the center of the image plane. An axis included in the horizontal plane and orthogonal to the above axis. This axis is also the coordinate axis of the u coordinate value. The axis and the axis orthogonal to the above axis. This axis is also the coordinate axis of the v coordinate value.

Next, coordinate conversion is performed on each of the unit vectors e1 ', e2', e3 'thus obtained, and the direction vector e1 and the direction vector e based on the world coordinate system 2 are used.
2. Find the direction vector e3. This conversion is performed in the following procedure.

First, when the rotation matrix for rotating about the z axis shown in the world coordinate system 2 by θ is represented as Rot (z, θ), and the rotation matrix for rotating about the x axis by ψ is represented as Rot (x, ψ), The rotation matrix RT, 1 for converting from the orthogonal coordinate system fixed to the TV camera 3 to the world coordinate system 2 is obtained by the following equation. RT, 1 = Rot (z, θz1) · Rot (x, θx1). ． ． (1)

Similar rotation matrices RT, 2 and RT, 3 are obtained for the TV cameras 4 and 5 as follows. RT, 2 = Rot (z, θz2) · Rot (x, θx2). ． ． (2) RT, 3 = Rot (z, θz3) · Rot (x, θx3). ． ． (3)

A rotation matrix RT, 1, a rotation matrix RT, 2, and a rotation matrix RT, 3 are calculated based on these equations (1) to (3). Then, the rotation matrix RT, 1 calculated in this way
˜RT, 3 is applied to the unit vectors e1 ′, e2 ′, e3 ′ according to the following equations (4) to (6), and the direction vector e1,
Find e2 and e3 respectively. e1 = RT, 1 · e1 '. ． ． (4) e2 = RT, 2.e2 '. ． ． (5) e3 = RT, 3.e3 '. ． ． (6)

Next, in step s8, the distance d1 from the TV camera 3 to the characteristic point P1 and the characteristic point P from the TV camera 4 are calculated.
The distance d2 to 2 and the distance d3 from the TV camera 5 to the feature point P3 are calculated by the procedure described below. First, a position vector OP1, a position vector OP2, and a position vector OP3 from the origin of the world coordinate system 2 to each of the feature points P1 to P3.
Is OP1 = r1 + d1e1. ． ． (7) OP2 = r2 + d2e2. ． ． (8) OP3 = r3 + d3e3. ． ． It is expressed as (9).

On the other hand, the distance between the feature points has already been given the information of distance data L3, j, distance data L2, j, distance data L1, j in step s6, so that | r2-r1 + d2e2-d1e1 | = L3, j. ． ． (10) | r3-r1 + d3e3-d1e1 | = L2, j. ． ． (11) | r3-r2 + d3e3-d2e2 | = L1, j. ． ． (12) The following equation holds. In step s8, the distances d1 to d3 are calculated by solving these equations.

Next, in step s9, the position vector rw of the characteristic point P1 in the world coordinate system 2, the displacement vector a from the characteristic point P1 to the characteristic point P2, and the displacement vector b from the characteristic point P1 to the characteristic point P3 are set. Formulas (13) to (1
Calculated according to 5).

Next, in step s10, it is determined whether or not the command sent from the robot controller 12 is a teaching command. In this case, the result is "Yes" and the process proceeds to step s11. Next, when proceeding to step s11,
The position vector rw and the displacement vectors a and b obtained in step s9 are used as the vehicle model No. The reference position data rwj, aj, bj corresponding to j is stored in the storage device 11. In this way, the reference position of the work 1 is taught, while the work to be performed on the work 1 at the reference position is taught to the robot. Next, in step s12, a detection end signal is output to the robot controller 12, and this routine ends.

<Actual Work> In the actual work after the teaching is finished, the vehicle type No. to be carried into the work space is determined based on the reference position data obtained as described above. A displacement of j from the reference position of the work 1 is detected, and the operation of the robot is corrected based on this displacement. The processing in this case will be described below.

When the work 1 is carried into the work space, the robot controller 12 causes the vehicle body No. A deviation amount measurement command is sent to the calculator 10 together with j. As a result, as in the case of the above teaching, first, step s1 to step s9 in FIG.
Find w and displacement vectors a and b. Next, step s10
If you proceed to, the judgment result will be judged as "No".
It proceeds to step s13.

In step s13, the storage device 11
The reference position data rwj, aj, bj are read from. Next, in step s14, the parallel displacement amount x and the rotation matrix R are calculated. The processing will be described below. First, the parallel displacement amount (vector) x is obtained by substituting the position vector rw calculated in step s9 and the position vector rwj in the reference data read in step s13 into the following equation (16). x = rw-rwj. ． ． (16) Next, the vector product of the displacement vectors aj and bj in the reference data read in step s13 dj = aj * bj. ． ． (17) is obtained. Further, the following unit vectors uj, wj, vj are obtained. uj = aj / | aj |. ． ． (18) wj = dj / | dj |. ． ． (19) vj = wj × uj. ． ． (20)

In this way, the basis vectors uj, vj, wj of the x, y, z axes of the coordinate system fixed to the work 1 with the characteristic point P1 as the origin and the direction of the displacement vector a as the x axis.
Is obtained. Work 1 using these basis vectors
Rj = (uj, vj, wj). A rotation matrix for coordinate conversion of the coordinate values corresponding to the coordinate system fixed to the above into coordinate values corresponding to the world coordinate system 2. ． ． (21) is obtained.

Similarly, the rotation matrix R0 is calculated from the position data rw, a, b. As a result, the rotation matrix R representing the amount of deviation is R = R0.Rj-1. ． ． It is calculated from (22). As described above, the parallel shift amount x and the rotation matrix R are calculated.

Next, when proceeding to step s15, step s
The parallel displacement amount x and the rotation matrix R calculated in 14 are output to the robot controller 12. Robot controller 1
2 corrects the operation of the robot based on these pieces of information.

C: Other Modifications Another modification is shown in FIGS. 5 and 6. In another modification shown in these drawings, the TV cameras 3 and 4 in the above-described embodiment are replaced with one TV camera 3 '. This is used when the positions of the two feature points are close to each other or when the accuracy of the above embodiment is not required. The image processing device 7'is T
The coordinates (ui, v) of the two feature points captured by the V camera 3 '
i) (i = 1, 2) is obtained and sent to the storage device 11. The subsequent processing procedure is the same as that of the above-described embodiment. in this case,
If the same processing speed as that of the above-described embodiment is required, the processing capacity of the image processing apparatus 7'may be set to about twice that of the image processing apparatus 7.

Further, all the characteristic points may be captured with one TV camera. As the characteristic point, the edge of the member is used in the above embodiment, but a hole or a protrusion such as a bolt may be used.

[0039]

As described above, according to the present invention, the coordinates of at least three characteristic points that are not on the same straight line on the work are obtained as data representing the position and orientation of the work. There is an effect that it is possible to realize a work position detecting device that can handle a work of any shape and that is low in price and capable of high-speed processing.

[Brief description of drawings]

FIG. 1 is an overall view of a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a first exemplary embodiment of the present invention.

FIG. 3 is a diagram showing an example of an image of feature points captured in the present invention.

FIG. 4 is a PAD diagram showing a processing procedure of the arithmetic unit 10 according to the first embodiment of the present invention.

FIG. 5 is an overall view of a second embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration of a second exemplary embodiment of the present invention.

[Explanation of symbols]

 1 Work 2 World Coordinate System (Reference Coordinates) 3, 4, 5 TV Camera (Photographing Means) 6 Controller 7, 8, 9 Image Processing Device (Detecting Means) 10 Calculator (Calculating Means) 11 Storage Device (Storage Means) 12 Robot controller 13, 14, 15 images.

Claims (2)

[Claims] 1. A photographing means for photographing an image of at least three feature points that are not on the same straight line on a work, and a detection for detecting coordinates of each feature point photographed by the photographing means in an image plane. A work position detecting device comprising: a means and a calculation means for calculating coordinates of these feature points in a predetermined reference coordinate system based on coordinates of the feature points in an image plane. 2. A storage means for storing the coordinates of each of the feature points in the reference coordinate system, and when the coordinates of each of the feature points of the new workpiece in the reference coordinate system are calculated, those coordinates and the storage means. 2. The work position detecting device according to claim 1, further comprising a position deviation detecting means for calculating a position deviation of a new work on the basis of the coordinates stored in.