JPH06270083A - Work position detector - Google Patents

Abstract

PURPOSE:To detect a position of a work moving over a wide range in real time with high accuracy. CONSTITUTION:TV cameras 2-4 photograph images of three characteristic points on a work, and image recognizing devices 10-12 detect respectively coordinates in an image plane of these respective characteristic points, and a computing device 13 carries out computation on the coordinates in a reference coordinate system of the respective characteristic points according to these coordinates, and sends a position or posture correcting command of the respective TV cameras to a driving means of universal heads 5-7 so that the respective characteristic points can fall within visual fields of the respective TV cameras 2-4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a work position detecting device suitable for use in an automatic production line device having a robot or the like.

[0002]

2. Description of the Related Art In an automatic production line, it is necessary for each production apparatus to perform an appropriate work on a work flowing in the line, and for that purpose, it is necessary to accurately detect the position and orientation of the incoming work. Conventionally, there have been the following two methods for detecting the position of such a work.

<First Method> It is assumed that the work is an object having 6 degrees of freedom capable of parallel movement / rotational movement, and its position / orientation is calculated. When the work position is detected, three points that are not on the same straight line on the work, such as holes, are defined as representative points. Then, each of these representative points is photographed by a stereo camera method in which two cameras are seen from different directions, and the position of the work in the work space and Ask for posture.

<Second Method> First and second sides that are not parallel to each other are selected from the respective sides that define the outer shape of the work, and the positional relationship between these two sides is stored in advance in a memory. . Then, when the work arrives in the work space, two intersecting points where the first side of the work intersects the mutually parallel first and second detection planes set in the work space are detected by, for example, optical means. To do. Further, the intersection point where the second side of the work intersects the third detection plane that is non-parallel to the first and second detection planes is detected. Then, based on the coordinates of these three intersections, position information regarding the first and second sides is obtained, and the position of the work is detected based on these position information. The applicant of the present application has proposed a work position detecting device based on this method in Japanese Patent Application No. 2-9541.

[0005]

However, in the first and second methods described above, since the work position is detected by photographing the work with a fixed camera or the like, the field of view of the camera or the like is detected. However, there is a problem that it is only possible to detect the work position within a narrow range corresponding to. Further, since the work position detectable range is narrow, there is a problem that the work position can be detected only for a short time when the work passes through the range. Here, if the field of view of the camera or the like is widened, it is possible to broaden the detectable range of the work position. However, when the field of view of the camera is widened, this causes a problem that the measurement accuracy of the work position is reduced.

The present invention has been made in view of the above circumstances, and provides a work position detecting device capable of detecting the position of a work flowing in a line over a wide range without lowering the measurement accuracy. The purpose is to provide.

[0007]

A work position detecting apparatus according to the present invention takes an image of a feature point on a work by a photographing means, and based on the image formation position of the feature point in an image plane, A work position detecting device for calculating coordinates of a feature point in a predetermined reference coordinate system and outputting the information as information for specifying the position and orientation of the work, wherein the feature point is photographed by the photographing means. It is characterized by comprising a changing means for changing a mounting position or a posture of the means.

[0008]

According to the above invention, since the position or posture of the photographing means is changed so that the characteristic points are photographed by the photographing means, it is possible to detect the position of the work moving over a wide range.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

A: Structure of Embodiment FIG. 1 is an overall view showing a work position detecting apparatus according to this embodiment installed in a certain process of an automobile production line. In this figure, O is a reference coordinate system fixed in the work space. This reference coordinate system is an orthogonal coordinate system composed of an X _O axis, a Y _O axis, and a Z _O axis, and is used to specify the position of the work in the working space in this embodiment. 1
Is a car body that is a work, and is conveyed into the work space of this process by a conveyor (not shown). Here, the transport direction of the vehicle body 1 substantially coincides with the X _O axis direction.

2, 3 and 4 are television cameras (hereinafter,
The image is abbreviated as a TV camera), and images of three feature points 9a, 9b, and 9c that are not on the same straight line on the surface of the work 1 are captured. Here, the TV cameras 2, 3 and 4 are configured so that their positions or postures are automatically changed in accordance with the movements of the feature points captured by the respective TV cameras (described later), so that their field of view is set to work. It is set relatively narrow regardless of the total movement of the workpiece in the space. For example, when the transport accuracy is relatively good, the size of the field of view is set to "± several tens cm".

In this embodiment, the position of each characteristic point in the working space (that is, the coordinate value in the above-mentioned reference coordinate system) is obtained by photographing each characteristic point by each TV camera 2, 3, 4. , The position of each feature point is obtained by converting the coordinates of each feature point on the image plane obtained by photographing into coordinates in the reference coordinate system. In order to make this coordinate conversion possible, this embodiment assumes an orthogonal coordinate system fixed to each TV camera. Here, the orthogonal coordinate system fixed to each TV camera means a coordinate system constituted by the following coordinate axes. z-axis: An axis corresponding to the shooting direction of the camera (direction in which the center of the field of view is viewed from the camera). All points on this axis are imaged in the center of the image plane. x-axis: An axis included in the horizontal plane and orthogonal to the z-axis. y-axis: An axis orthogonal to the z-axis and the x-axis.

FIG. 2 illustrates an image around the feature point 9a taken by the TV camera 2. R in this figure
(0,0) represents the center of the image plane. In the illustrated case, the feature point 9a is located at a position P separated from the center by u pixels and v pixels in the x-axis direction and the y-axis direction, respectively.
The image is formed at (u, v). In this embodiment, the position of each feature point in the reference coordinate system is calculated based on the shift amount data (u, v) representing the distance from the screen center for each feature point obtained via each TV camera.

Reference numerals 5, 6, and 7 are pan heads, and the TV cameras 2, 3, and 4 are fixed to these pan heads. These pan heads 5, 6 and 7 are provided with drive means such as servo motors, and by these drive means driving each pan head in accordance with a command from the controller 8, the position of each TV camera or The posture is changed, that is, the TV cameras are moved in parallel or the vertical angle and the horizontal angle are changed. The control for causing the above-mentioned TV camera to follow the movement of the characteristic points is also performed by the controller and the driving means.

Next, the system configuration of this embodiment will be described with reference to the block diagram shown in FIG. As shown in the figure, the controller 8 includes image recognition devices 10, 11, 1
2, the calculator 13 and the storage device 14. First, the image recognition devices 10, 11 and 12 respectively detect the feature points 9a, 9b and 9c based on the images captured by the TV cameras 2, 3 and 4, and determine the position of each feature point on the image plane. The shift amount data (u, v) representing the above is output to the arithmetic unit 13, respectively. The storage device 14 stores the distances between the respective feature points of the work 1, that is, the distances between the feature points 9a and 9b, the distances between the feature points 9b and 9c, and the feature points 9c and 9.
The distance a is stored as distance data. Generally, the distance data varies depending on the vehicle type of the vehicle body of the work 1. Therefore, the storage device 14 is preliminarily stored with distance data corresponding to all vehicle types handled on the production line, and by designating a vehicle type number, the distance data corresponding to the vehicle type number is read out. In addition, the storage device 14 includes the above-mentioned TV camera 2,
Data regarding the installation conditions 3 and 4, that is, the following data are stored. Coordinate values of the installation positions of the TV cameras 2, 3 and 4 in the reference coordinate system O Horizontal angle and vertical angle of each shooting direction at the initial positions of the TV cameras 2, 3 and 4 Based on the deviation amount data (u, v) A relational expression for calculating the movement amount in the horizontal angle rotation direction and the movement amount in the vertical angle rotation direction of the TV camera.

The computing device 13 stores the distance data corresponding to the vehicle type number of the work 1 stored in the storage device 14 and the above-mentioned T
Data concerning the installation conditions of the V cameras 2, 3, 4 and shift amount data (u, v) corresponding to each feature point supplied from the image recognition devices 10, 11, 12 and the pan heads 5, 6, 7 Based on the current position, horizontal angle and vertical angle,
Data for specifying the position and orientation of the work 1 in the reference coordinate system is calculated and output to the main controller 15. The main controller 15 includes a computing device 13
Based on the data obtained from the above, a signal for controlling a robot (not shown) is output. In addition, the arithmetic unit 13 calculates the correction amount of the horizontal angle or the vertical angle of each TV camera 2, 3, 4 necessary to put each feature point in the field of view of each TV camera 2, 3, 4 for each shift amount data. It is calculated based on (u, v) and the above relational expression, and a command for correcting the horizontal angle or the vertical angle according to the correction amount is given to the drive means of each TV camera.

B: Operation of Embodiment Next, the operation of this embodiment will be described with reference to FIGS. 4 and 5. First, in step S1 shown in FIG. 4, the vehicle type number of the work 1 and an instruction to start the position detection of the work 1 are sent from the main controller 15 to the arithmetic unit 13. The computing device 13 reads from the storage device 14 the distance data between the respective characteristic points corresponding to the vehicle type number and the data (including the above data) regarding the installation conditions of the TV cameras 2, 3 and 4. Further, the recognition flag (described later) and the detection flag (described later) are turned off to initialize the data.

Next, in step S2, the arithmetic unit 13
Creates a command to move the TV cameras 2, 3, 4 to the initial position based on the data read in step S1 and gives the command to each drive means of the platform heads 5, 6, 7 to determine the position of each TV camera. To initialize. Next, in step S3, the arithmetic unit 13 starts a timer for measuring a fixed time.

Next, in step S4, it is determined whether or not any of the following conditions is satisfied, and the result of this determination is "Y".
During the period of "es", each processing of steps S5 to S17 for detecting the position of the characteristic point, controlling the position or posture of the platform, etc. is executed. Condition 1: Within a fixed time after the timer is started Condition 2: All recognition flags corresponding to each TV camera 2, 3 and 4 are in ON state

In the above description, the recognition flag means the TV.
This is a flag that is turned on when the camera detects a feature point. The detection flag is a flag that is turned on when all the recognition flags are in the on state. The reason why the condition 1 or 2 is satisfied in step S4 is as follows. That is,
To detect the position of the work 1, all the feature points 9a, 9
b and 9c must be captured by the TV cameras 2, 3 and 4, but if these characteristic points deviate significantly from the field of view of the TV camera, the work 1 passes through the work space without being detected in position. It is possible that there will be problems. Therefore, if the time during which all the characteristic points are not recognized exceeds a certain time, it is considered as a symptom of the above-mentioned defect, and whether the above condition 1 or 2 is met to generate an alarm for dealing with this defect. Is making a decision.

Next, each processing of steps S5 to S17 will be described. First, the determination result of step S4 is “Yes
When it becomes "s" and proceeds to step S5, the camera number n is changed from "1" to "3" (n = "1" to "3" is T
V cameras 2 to 4 and image recognition devices 10 to 12 respectively), and steps S6 and S7 are executed for each n. First, in step S6, the arithmetic unit 13 gives an instruction to capture an image to one of the image recognition devices 10 to 12 corresponding to the camera number n. As a result, the image recognition device corresponding to the camera number n captures the image from the corresponding TV camera, and compares the density of each pixel forming the image with a predetermined threshold value to form a black-and-white pattern of the image,
The image of the feature point is detected from the black-and-white pattern thus obtained. Further, when the image recognition device can detect the feature point, the image recognition device obtains the shift amount data (u, v) for the feature point. Next, in step S7, the arithmetic device 13 determines whether or not the image recognition device corresponding to the camera number n has detected a feature point. This judgment result is "Ye
If it is "s", step S8-showing the PAD diagram in FIG.
The process proceeds to each process up to S14. On the other hand, when the result of this determination is "No", the recognition flag corresponding to the camera number n is turned off (step S15), and the process returns to step S5.

Next, when proceeding from step S7 to step S8 shown in FIG. 5, the arithmetic unit 13 turns on the recognition flag corresponding to the camera number n. Next, in step S9, the feature point deviation amount data (u, v) is fetched from the image recognition device corresponding to the camera number n. Next step S1
In step S9, the shift amount data (u, v) obtained in step S9 is converted into a vertical movement direction or a horizontal movement direction of the platform according to the above relational expression stored in the storage device 14. Next, in step S11, step S1
The target position of the platform is calculated according to the following equation using the movement amount calculated in 0. Target position of pan head = current position of pan head +
Amount of movement

Next, in step S12, step S1
It is determined whether or not the target position obtained in 1 is outside the operable range of the platform. If the result of this determination is "Yes", then the operation proceeds to step S13, in which the target position of the platform is set to a value within the operable range that is closest to the target position obtained in step S11, and so is set. The amount of movement is corrected according to the desired target position, and step S1
Go to 4. On the other hand, the determination result of step S12 is “No”
If so, the process proceeds to step S14 without executing step S13. Next, in step S14, the platform corresponding to the camera number n is driven to the target position calculated as described above. Then, step S in FIG.
Returning to step 5, the above processing is repeated for a new camera number n, and when the above processing is completed for all camera numbers n = “1” to “3”, the process proceeds to step S16.

Next, in step S16, it is determined whether or not all the feature points 9a, 9b, 9c have been recognized by the TV cameras 2, 3 and 4, that is, the camera number n = "1" to "3".
It is determined whether all the recognition flags corresponding to are in the ON state. If the result of this determination is "Yes", the flow proceeds to step S17, where the positions of the respective characteristic points 9a, 9b, 9c on the image plane captured by the TV cameras 2, 3, 4 and the storage device 14 are stored. Based on the distance data between the respective feature points of the work 1 and the current position and posture (vertical angle and horizontal angle) of the TV cameras 2, 3 and 4,
The position of each feature point in the reference coordinate system is calculated, and the work 1
Is output to the main controller 15 as information for specifying the position and orientation of the. A method for converting the coordinate value of each feature point on the image plane into the coordinate value in the reference coordinate system in this way is described in Japanese Patent Application No.
No. 7967. Further, the process proceeds to step S18, and the detection flag is turned on because all the TV cameras 2, 3, 4 are turned on.

On the other hand, the determination result of step S4 is "No".
If so, the process proceeds to step S19, and the timer is stopped and reset. Next, in step S20, it is determined whether or not the detection flag is in the ON state. When the detection flag is in the OFF state, that is, when the work position cannot be detected by the work position detection device, the determination result is “No”, the process proceeds to step S21, and the work is sent to the main controller 15. Notify the detection error.

C: Effects of Embodiments According to the above embodiments, the following effects can be obtained. (1) The position of a work moving over a wide range can be detected in real time and with high accuracy. (2) Since the image processing performed by the image recognition device or the like may be as simple as identifying the position of the characteristic point, the processing speed is increased and the efficiency of the production line can be improved. (3) Since each characteristic point may be three points that are not on the same straight line, there are few restrictions on the shape of the work.

<Modification> In the above embodiment, the three feature points 9a, 9b, 9c are photographed by three TV cameras. However, when the feature points are close to each other, two feature points 9 or 9 are used.
It is also possible to capture and process individual feature points. In the case where one camera captures two feature points among these cases, when the shift amount of each feature point is obtained in step S9, the midpoint of the two feature points and the center of the image plane are determined. The moving amount of the pan head may be determined so that Also, when three feature points are captured by one camera, x,
The amount of movement of the platform may be determined so that the maximum value and the minimum value of the y coordinate values are equal.

[0028]

As described above, the work position detecting apparatus according to the present invention takes an image of the vicinity of the feature point on the work by the photographing means, and based on the image formation position of the feature point in the image plane. A work position detection device for calculating coordinates of the feature point in a predetermined reference coordinate system and outputting the information as information for specifying the position and orientation of the work, wherein the feature point is photographed by the photographing means. Since the changing means for changing the mounting position or the posture of the photographing means is provided, there is an effect that the position of the work moving over a wide range can be detected in real time and with high accuracy.

[Brief description of drawings]

FIG. 1 is an overall view of an embodiment of the present invention.

FIG. 2 is a diagram showing an example of an image of feature points captured in the same embodiment.

FIG. 3 is a block diagram showing a system configuration of the embodiment.

FIG. 4 is a PAD diagram explaining the operation of the embodiment.

FIG. 5 is a PAD diagram explaining the operation of the embodiment.

[Explanation of symbols]

 1 Work O Reference Coordinate System 2,3,4 TV Camera (Shooting Means) 5,6,7 Platform (Changing Means) 10,11,12 Image Recognition Device 13 Computing Device 14 Storage Device 15 Main Controller

Claims (1)

[Claims] 1. An image near a feature point on a work is photographed by a photographing means, and a coordinate of the feature point in a predetermined reference coordinate system is calculated based on an imaged position of the feature point in an image plane, A work position detecting device for outputting as information specifying the position and posture of the work, comprising a changing means for changing the mounting position or posture of the photographing means so that the characteristic points are photographed by the photographing means. A work position detection device characterized by the above.