JPH0665472B2 - Visual device - Google Patents

Description

TECHNICAL FIELD The present invention relates to a visual device, and is particularly useful as a visual sensor for an industrial robot.

BACKGROUND ART Conventionally, as a visual device used for an industrial robot,
It is known that a two-dimensional image obtained from one television camera is processed by a differentiation method or a region method, or a gradation process is performed.

There is also known a visual device that obtains three-dimensional information by a so-called light emitting method using one camera and a projection device of light having a special shape (for example, slit light).

Further, a visual device is known in which two television cameras are used to perform stereoscopic vision from corresponding points on a two-dimensional screen obtained by each camera and the arrangement of both cameras.

Problems of Prior Art With the sophistication of industrial robots in recent years, sophistication has also been required as a visual device.

For example, in the case of an assembly robot, conventionally, since a relatively simple work is performed in which a single type of part placed in a certain case is picked up and attached to an object, a simple visual system is sufficient. It was

However, recently, robots have been advanced so that specific parts can be searched from randomly placed disparate parts, and they can be picked up and attached to a target object. There is a demand for a high degree of sophistication so that specific components can be found out from inside.

In addition, in the case of a mobile robot that requires matching with a taught environment model and obstacle avoidance sensing, the sophistication of a visual device is also required.

In addition, even if the position of the object varies greatly because it is flown on the line, it is required to have a sophisticated visual device that can be applied to an inspection system that can observe the object from a predetermined target position with respect to the object. Is becoming.

However, there is a problem in that the conventional visual device cannot sufficiently meet such a demand for sophistication.

Object of the invention It is an object of the present invention that stereoscopic viewing can be performed by one camera means, and in addition to the stereoscopic viewing, it can be viewed from different eyes, or a desired portion can be enlarged.
It is an object of the present invention to provide a visual device capable of performing multifaceted visual recognition such as seeing a hidden part and capable of handling advanced robot work.

The visual device of the present invention includes a camera means, a moving means of the camera means, a stereoscopic visual recognizing means for performing stereoscopic vision by moving the camera means to a first position and a second position by the moving means, and the stereoscopic means. A reference feature selecting means for extracting and selecting a reference feature of an object based on visual data; an auxiliary position calculating part for calculating an auxiliary position based on the position of the reference feature; and the camera means. It is a structural feature that it is provided with an auxiliary visual recognition means which is moved and visually recognized from the auxiliary position.

Examples Hereinafter, the present invention will be described in more detail based on the examples shown in the drawings. Here, FIG. 1 is a schematic block diagram of an embodiment of the visual device of the present invention, FIGS. 2 and 3 are perspective views of a situation in which stereoscopic vision is performed by the visual device shown in FIG. 1, and FIG. FIG. 1 is a perspective view showing a situation in which the visual device is viewed from different viewing angles, FIG. 5 is an illustrative view of a two-dimensional image obtained in the state of FIG. 4, and FIG. 6 is a conceptual explanation of the viewing angle by the device of the present invention. FIG. 7 and FIG. 7 are conceptual explanatory views of the stereoscopic view and the moving view. The present invention is not limited to the embodiments shown in the drawings.

As shown in FIG. 1, the visual device 1 includes a camera 3, a camera arm 4 for moving the camera 3, a camera control unit 5, a preprocessing unit 7, an addressing unit 8, and an image memory 9. It is basically provided with a microprocessor 10, a viewing angle setting unit 11, an environment model pattern storage unit 12, and an arm control unit 13. 14 is a control bus,
15 is a data bus.

The camera 3 can be moved by the camera arm 4.

The image data obtained from the camera 3 is subjected to preprocessing such as spatial differentiation, smoothing and edge enhancement in the preprocessing unit 7, and is taken into a predetermined location of the image memory 9 designated by the address designating unit 8.

The control for capturing an image is performed by the camera control unit 5 based on a command given by the microprocessor 10 via the control bus 14.

The microprocessor 10 reads image data from the image memory 9 via the data bus 15 and performs image processing such as feature extraction and distance calculation.

The viewing angle setting unit 11 uses the environment model pattern storage unit 12 or the environment model pattern storage unit 12 based on data taught in advance according to the purpose of work in order to perform stereoscopic viewing and visual recognition from a special viewing angle.
The moving position and posture (line-of-sight direction) of the camera are set based on the condition judgment such as comparing and collating the environment model pattern (or visual model pattern) set in advance with the image data obtained from the camera 3.

Arm control unit 13, based on the output from the viewing angle setting unit 11,
The camera arm 4 is driven by performing necessary coordinate conversion and drive control.

Now, as a specific example, assuming the case where a specific part is taken out by a robot from among parts of different types placed, the operation of the visual device 1 as the visual sensor of the robot will be described. That is, as shown in FIG. 2, consider a work in which parts a, b, c, d of different types are randomly placed, and the part a is taken out by the robot R from them.

The microprocessor 10 drives the camera arm 4 via the view angle setting unit 11 and the arm control unit 13, and the camera arm 4 is driven by the microprocessor 10 as shown in FIGS.
The camera 3 is positioned at the first position shown in the figure to obtain the first image data. Next, as shown in FIG. 3, the camera 3 is positioned at a second position separated from the first position F ₁ by a predetermined distance, and a field of view overlapping the field of view is observed from the first position F.

Since the visual fields from both positions overlap, one corresponding part is included in each image data obtained at both positions, but there is a parallax due to a predetermined positional difference. Therefore, based on this parallax, the microprocessor 10 uses the parts a,
Three-dimensional visual observation, that is, stereoscopic vision of b, c, and d is performed.

Since the microprocessor 10 is given the shape data of the desired part a in advance, if the necessary shape data is obtained from the image data, the target part a can be recognized, but as shown in FIG. Since the parts a to d hide each other from each other, necessary data is not always obtained by the stereoscopic view.

In such a case, the microprocessor 10 calculates and estimates the position and orientation of the target part a by extracting and selecting the reference characteristic part of the target part a from the obtained image data. This is possible even if the target part a is partially hidden. The reference feature portion can be extracted by, for example, a differential method, and can be selected, for example, based on the perspective of the camera and the degree of bending of the surface. Then, the auxiliary position at which the target part a can be observed is calculated from the relative position with respect to the target part a taught in advance. Then, the viewing angle setting unit gives an instruction to perform the moving view.
Output to 11.

The viewing angle setting unit 11 drives the camera arm 4 according to the instruction, and moves the camera 3 to the auxiliary position.

As a result, the microprocessor 10 obtains image data viewed from a predetermined relative position.

FIG. 4 shows this state, in which the camera 3 is moved by the camera arm 4 and the parts a to d are observed from directly above.

The image data obtained from the position shown in FIG. 4 is, for example, as shown in FIG. 5, so that the data of the portion which cannot be obtained by the stereoscopic view from the camera position shown in FIGS. 2 and 3 can be obtained. Has become. For example, the part of the part a that was not visible because it was hidden by the part b can actually be visually recognized.

Thus, the microprocessor 10 can obtain necessary and sufficient data on the parts a to d, recognize the part a, drive the robot R based on the result of the visual recognition, and take out the target part a without error. .

As an example of the example, by extracting and selecting the reference feature portion on the object by stereoscopic vision, calculate the position in stereoscopic view of the target portion of the object that was previously taught at the target position for the reference feature point, An example is one in which a camera is brought close to the position to perform a magnified view. In this case, first, the position of the camera may be changed so as to capture the target portion in the center of the screen, and then the camera may be moved forward, or the posture may be changed while moving forward.

As still another embodiment, the data obtained in stereoscopic vision and the data taught in advance are compared with reference to time or the position of the camera, and when only a correlation less than a predetermined value is obtained, a certain amount is present in the visual field. One that determines that there is an abnormality and moves the camera to perform auxiliary visual recognition is included. In this case, an alarm may be output or the operation of the robot or line may be stopped for safety.

6 and 7 generally describe the stereoscopic view and the moving view.

As an example of the object, there is a hollow tetrahedron with one face, and when this is captured from two positions, the camera position F ₁ initially set and the camera position F ₂ moved by a predetermined distance, F ₁ and F ₂ An image of a tetrahedron is obtained on the visual field planes V ₁ and V ₂ and is stored in the image memory.

If the distance between F ₁ and F ₂ is known and the corresponding points P ₁ and P ₂ on the image of the reference feature point P on the tetrahedron are identified, stereoscopic vision including distance measurement becomes possible. This is known.

Capture the main image of the tetrahedron at the reference position F ₁ and then turn the camera
For example, with a moving mechanism such as a robot arm, an XY stage, or a manipulator, for example, when a tetrahedron is viewed from the side, it is at the position F _{3 in} the figure, and when it is magnified in detail in a certain viewing direction, it is the position F _{4 in} the figure. Move to. In addition, like the mobile robot, the line-of-sight rotation is performed when looking at the desired environment model or confirming the direction for avoiding obstacles.

The movement control of these cameras is performed by moving to a position taught in advance, or moving to a position where a required visual image according to a work purpose is obtained by programming a conditional judgment function.

For example, the camera is moved by controlling the arm by setting a predetermined position / posture of the movement destination with the reference position F ₁ as the origin of the robot coordinate system.

To set the position, select the feature point P on the screen of F ₂ (or F ₁ ).
Determine the desired line-of-sight direction and visual distance based on ₁ (or P ₂ ).

Since this gaze direction cannot be directly set in the 3D direction on the 2D screen, the user proceeds to the 2D position / orientation set on the screen to check the corresponding point P ₃ , and further sets the direction on the F ₃ screen.

The line-of-sight direction is performed while confirming the corresponding points of the reference in the visual field, so that the arm control can be performed by the information feedback of the moving vision.

Thus arm coordinates (X, Y, Z, θ ) is a reference point (x _2,
y ₂ ), the movement of the arm can be controlled by setting the visual information of the desired visual position (gaze direction, visual distance), but when the reference feature point cannot be obtained, for example, when the target is a cylinder or a sphere, the visual Arm control is provided by visual information feedback that sets the distance and compares it to the storage of the predicted shadow image pattern.

As can be understood from the above description, according to the visual device 1, a stereoscopic view can be obtained by moving the camera, the overall situation can be grasped by the stereoscopic view, and then the stereoscopic view can be performed by the moving view at a position other than the stereoscopic view. Obtaining visual information that cannot be obtained by stereoscopic vision,
As a result, since the robot R can be controlled, the robot R has an advanced visual function capable of coping with the advanced work of the robot R.

EFFECTS OF THE INVENTION According to the present invention, a camera means, a moving means of the camera means, a stereoscopic vision means for performing stereoscopic vision by moving the camera means to a first position and a second position by the moving means, and the stereoscopic vision. The reference feature part selecting means for extracting and selecting the reference feature part of the object based on the data in step 1, the auxiliary position calculating part for calculating the auxiliary position based on the position of the reference feature part, and the camera means are moved. There is provided a visual device characterized by comprising auxiliary visual recognition means for visually recognizing from the auxiliary position, whereby stereoscopic vision can be performed, and in addition, images from different viewing angles are provided. Since it is possible to obtain data, magnified view, image data of hidden places, etc., it is extremely useful as a visual device for advanced robots.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the visual device of the present invention, FIGS. 2 and 3 are perspective views of a situation in which stereoscopic vision is performed by the visual device shown in FIG. 1, and FIG. FIG. 1 is a perspective view showing a situation in which the visual device shown in FIG. 1 visually recognizes from different viewing angles, FIG. 5 is an exemplary view of a two-dimensional image obtained in the state of FIG. 4, and FIG. 6 is a conceptual explanatory view of the viewing angle by the device of the present invention. , FIG. 7 is a conceptual explanatory view of the stereoscopic view and the moving view. (Explanation of symbols) 1 ... Vision device 3 ... Camera, 4 ... Camera arm 9 ... Image memory 10 ... Microprocessor 11 ... Viewing angle setting unit 13 ... Arm control unit

Claims (1)

[Claims] 1. Camera means, means for moving the camera means,
A stereoscopic visual recognition means for moving the camera means to the first position and the second position by the moving means for stereoscopic vision, and a reference characteristic part for extracting and selecting a reference characteristic part of the object based on the data in the stereoscopic vision. It comprises a selection means, an auxiliary position calculation part for calculating an auxiliary position based on the position of the reference characteristic part, and an auxiliary visual recognition means for moving the camera means for visual recognition from the auxiliary position. A visual device characterized by being.