JPS60123974A - Multistage visual field recognizing system - Google Patents

Abstract

PURPOSE:To improve the speed of visual retrieval process of supplying parts and its recognizing accuracy, by switching the visual field of the visual sensation of a robot in multiple stages for every desired extent in accordance with the vertical and horizontal movements of the robot. CONSTITUTION:The image of the flow of plural kinds of parts is taken with a TV camera at a fixed time interval. A recognition process is first performed by a robot control section and visual sensation recognizing section with a wide visual field whether or not object parts exist in the visual field. When the existence of the object parts is confirmed, a robot is operated and a narrow visual field in set so as to make recognition process on the kind, location, and attitude of the object parts.

Description

[Detailed Description of the Invention] [Field of Application of the Invention] The present invention switches the field of view into fine and coarse stages so as to satisfy both the high-speed exploration function and the high-accuracy recognition function required for robot vision. This paper relates to a multi-stage field of view recognition method for recognizing multiple types of parts.

[Background of the invention]

Conventionally, when assembling parts using a robot, a dedicated feeding device for mechanically feeding the parts at a fixed position and posture has been required for each part. Instead, a general-purpose parts supply device with visual position and orientation recognition functions is configured.
A system that picks up and assembles parts within the robot's degree of freedom is considered suitable for high-mix, low-volume production.

As such a general-purpose parts supply device, a system has been considered in which parts are fed in bulk by a belt conveyor or a general-purpose parts feeder, and the parts are recognized by a visual device attached to the top. In this case, when the robot recognizes that the target part has been supplied in a position that can be grasped by the robot, the belt conveyor or parts feeder is stopped, the part is recognized again in a stationary state, and the correct position and orientation are determined. Send information to the robot.

The robot picks up parts and processes them according to visual information. Here, 2
This involves recognizing the degree of movement, but it is assumed that when the flowing part is checked and stopped, the part is still within the field of view.

However, if the number of target parts is increased to multiple types, the supply speed must be increased because it is necessary to quickly find the target parts (those in a position that the robot can grasp) from a mixed group of parts. The need arises, and the fixed vision system cannot handle this unless a wide field of view is set.

However, in order to accurately recognize the position and orientation of a component, it is necessary to increase the magnification with a narrow field of view. As described above, there are many difficulties in providing both position and orientation recognition functions, and the current situation is that a fully satisfactory practical device has not yet been realized.

[Purpose of the invention]

In view of the above-mentioned circumstances, an object of the present invention is to provide a multi-stage visual field recognition method that can speed up the visual search process for supply parts and improve the recognition accuracy in assembly processing using a robot/visual combination system. Our goal is to provide the following.

[Summary of the invention]

The multi-stage field of view recognition method according to the present invention provides a robot/visual field combination system that includes at least a robot equipped with vision, a robot control unit, and a visual recognition processing unit. The visual recognition processing unit first performs recognition processing to determine whether or not the target part exists within the same field of view using a wide field of view, and then when the existence of the target part is confirmed. The robot is operated to set a narrow field of view, and the type, position, and orientation of the target part are recognized.

The following is a supplementary explanation.

In other words, when it comes to the function of searching for parts, it is more effective in terms of speed to check the presence or absence of the target part by performing search image processing as few times as possible while keeping a field of view as wide as possible. In order to improve accuracy, it is more effective to set an expanded field of view as narrow as possible. In order to meet this requirement with one TV camera,
As will be described later, a TV camera is attached to the robot, and the field of view is switched according to the robot's movements to perform multi-stage search and recognition. In addition, subsequent assembly is performed by searching for parts from the above list, thereby omitting image search processing and speeding up visual processing.

[Embodiments of the invention]

Embodiments of the present invention will be described below with reference to the drawings.

First, Fig. 1 is a schematic external view of an embodiment of a robot/visual combination system to which the multi-stage visual field recognition method according to the present invention is applied, Fig. 2 is a partially enlarged view thereof, and Fig. 3 is a It is a flowchart of the system operation.

This system consists of a robot, a TV camera (vision and nine)
, robot control section, visual recognition section 1 Desired parts are assembled into the chassis flowing on the conveyor on the base machine by a parts supply device (such as a general-purpose parts feeder or magazine supply device).

Visual processing involves first capturing images of the flow of multiple types of parts being supplied at regular time intervals in the overall field of view shown in Figure 2(a), and then using the image processing to determine whether the required parts are within the field of view. Check if it does. If the target part is not found, the imaging 1 image processing is repeated at fixed time intervals until the target part is found. As shown in Fig. 2(a), if a general-purpose parts feeder or the like is used as a parts feeding device and there is a flow of parts, the viewing position may be fixed, but as shown in Fig. If the parts are placed side by side in the magazine, it is also necessary to move the field of view horizontally to find the parts.

A series of system operations for recognizing and assembling parts on the parts feeder will be explained with reference to FIG.

When a part is found in the entire field of view, the parts feeder is stopped, and the data of one or more parts found within the field of view is stored. The table below shows − of this parts list (list).
Give an example.

In this example, four parts are found within the field of view, and the parts are two a, one b, and one c (part types are roughly determined by area determination. Therefore,
Recognition errors are also taken into account. ), the approximate coordinates of the center of gravity of each part are determined.

However, instead of the center of gravity position, feature points of the part may be used instead.

From this parts list, search for the next target part to be installed. For example, when creating the parts list for the first time, the selection flags are all 0, indicating that none of the parts are selected, so one of them is selected. That is, one is always found at the beginning, and that is the entire visual field recognition end condition. Once selected, set the selection flag to 1
Then, the process moves on to recognizing the part with high accuracy.

In the cono processing, an enlarged field of view (wide field of view) is set according to the position of the part written on the parts list, as shown in FIG. 2(b), in order to enlarge this part and fit it within the field of view.

This is done by lowering and horizontal movements of the robot. After setting the enlarged field of view, accurate type identification and position/attitude recognition of parts is performed visually using a TV camera. If it turns out that the part is not the one you need, you go back to searching the parts list again. After confirming that the part is a necessary part, information on the position and orientation of the part is sent to the robot, and based on that information, the robot picks up the part and assembles it into the 7-palm.

Next, the system determines whether the next part to be installed is the next one and searches for the parts list again. If there is a corresponding part registered on the parts table and the selection flag is 0 and the part has not yet been assembled, the process moves to a recognition operation using an enlarged field of view for that part, and the same procedure is followed thereafter. If there are no more parts to be installed next in the parts list, the overall field of view is set again, the parts supply operation is started, the process moves to searching for parts, and the same procedure is followed.

Next, FIG. 4 is a block diagram of an embodiment of a robot/visual combination system to which the multi-stage visual field recognition method according to the present invention is applied, and FIG. 5 is a block diagram of its visual processing section.

Here, reference numeral 401 denotes a robot control unit which controls a series of operations such as selection of parts to be assembled into the reed and chassis, setting of visual field, visual recognition processing instructions, and assembly of parts by the robot.

That is, under the control of the robot control unit 401, the robot 403 moves to the full field of view position and the component supply device 405 is activated.

Subsequently, in response to instructions from the robot control unit 401, the visual recognition unit 402 performs full field of view recognition, and when recognition is completed, all parts supply devices 405 are stopped to create a parts table. Still, the robot controller 401 searches the parts list for the part specified by the robot controller 401, calculates positional information for setting the enlarged field of view from the position of the part, and sends the robot controller 4
Pass it to 01.

The robot control unit 401 controls the robot based on the information.
403 to set an enlarged field of view.

Next, according to instructions from the robot control unit 401, the visual recognition unit 402 recognizes the enlarged field of view, obtains accurate position and orientation information of the part, and calculates coordinates for the robot 403 to grasp the part from that information. The data is sent to the robot control unit 401.

Based on the information, the robot control unit 401 controls the operation of grasping the target part and assembling it on the shear/shear, and then selects the part to be assembled next and instructs the visual recognition unit 402 to select the part to be assembled next. The visual recognition unit 402 first searches on the parts list,
The presence or absence of the target part is notified to the robot control unit 401, and if the part is found, the enlarged field of view setting information for that part is sent to the robot control unit 401. If no parts are found, the robot control unit 401 starts up the parts supply device 405 again and restarts the above operation.

Finally, a more detailed recognition operation will be explained based on FIG.

The visual control section 500 controls the entire visual processing section.

The video input section 501 inputs the video signal from the TV left camera 04 and digitizes it, and the binarization processing section 502 performs the binarization processing. Next, the component area identification unit 503
This divides the binary image into areas for each group of black and white pixels, and records the part area included in the background area and the hole area included in the part area with identification.

Shape feature extraction units A and B (504, 505) each perform ring segment extraction and one ring line segment approximation of a component region.

Extracts small amounts of shapes such as area, center of gravity, principal axis of inertia, and various distances. The contents of these processes differ somewhat depending on the purpose of searching for the target part and the purpose of recognizing the position/orientation.

The shape recognition unit 506 compares the feature amount calculated by the shape feature amount extraction unit A (504) with the dictionary data A (513), and determines whether the part is the target part and the posture is such that the robot 403 can grasp it. Performs search processing. If the target part is found, the visual control unit 500 stops the parts feeder and the recognized shape registration unit 50
8 registers recognized shape data. Target shape selection section 51
0 selects a target part candidate from the recognized parts list created here, and the field of view setting instruction unit 512 creates data for setting the field of view to accurately recognize this part, and sends it to the robot control unit. 401.

After setting the field of view, the image data is input again, the feature amount is extracted by the shape feature extraction section B (505), and the feature amount is extracted by the shape recognition section B (505).
07), the part is checked against the dictionary data B (514) and the position and orientation of the part is recognized. Next, the robot positioning data creation unit 512 creates robot positioning data for grasping the target part, and transmits it to the robot control device 401 through the transmission processing unit 513.

〔Effect of the invention〕

As described in detail above, according to the present invention, it is possible to speed up the visual operation of supplied parts and improve the accuracy of its recognition, thereby improving the efficiency and performance of robot work. Remarkable effects can be obtained.

[Brief explanation of the drawing]

Fig. 1 is a schematic external view of an embodiment of a robot/visual combination flexible assembly system to which the multi-stage field of view recognition method according to the present invention is applied, Fig. 2 is a partially enlarged view of the system, and Fig. 3 is the same system. FIG. 4 is a block diagram of the operation flowchart, and FIG. 5 is a block diagram of the visual processing section. 401... Robot control unit, 402... Visual recognition unit, 403... Robot, 404... TV camera, 4
05...Component supply device. Agent Patent attorney Kosaku Fukuda (and 1 other person) z3 Figure Kaya 4 Continuation of the first page ■Int, CI,' Identification code Office serial number // G
01 B 11100 7625-2F11/28
8304-2F

Claims (1)

[Claims] 1. A robot/visual combination system consisting of at least a robot equipped with vision, a robot control unit, and a visual recognition processing unit. The visual recognition processing unit first performs recognition processing to determine whether or not the target part exists within the same field of view using a wide field of view, and then confirms the existence of the target part. When this happens, operate the robot to set a narrow field of view, and determine the type of target part,
Multi-stage field of view recognition method for position and orientation recognition processing. 2. In the item described in claim 1, when the existence of the target part is confirmed in a wide field of view, data on the type and position of parts other than the target part are stored, and then another target part is detected. When searching for a target part, first search for data related to the target part from among the above stored data, and then
When the presence of the object can be confirmed, the multi-stage field of view recognition method shifts to recognition using a narrow field of view, and when the presence cannot be confirmed, the process starts with recognition using a wider field of view.