JPH05228866A - Controller of automatic holding device in use of visual sense - Google Patents

Abstract

PURPOSE:To make a parameter unitarily manageable by storing the parameter required for the holding operation and image processing of a work in the same storage part as a program comprising both image processing and holding operation commands, and transferring it to an image processing part from this storage part at need. CONSTITUTION:A controller is provided with a robot action control part 62 controlling four axial motors 6a-6d of a robot and a motor 6e of a shuttle, a hand action control part 63 controlling three axial motors 22a-22c of a hand, and an image processing part 64 controlling each of driving motors 4a-4b of an XY table and a camera 2, and so on. In this case, there is provided a command interpretive part 87 which performs the interpretive execution of a command pertaining to a robot program. On the other hand, there is also provided a common memory 60 being held each in common among both these action control parts 62, 63, the image processing part 64 and the common interpretive part 87. In succession, a program comprising both image processing and holding operation commands of a work and a parameter for calculating position information of the work are stored in this common memory 60.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a control device for recognizing a gripping position of a work to be gripped and controlling an automatic gripping device for gripping the work.

[0002]

2. Description of the Related Art A laborious task in controlling an automatic assembly apparatus is teaching of position coordinates of a work or the like. Since the robot can move hands and fingers only to the taught position, if the teaching position is incorrect or incorrect, the assembly operation will fail or the hand and fingers will be destroyed. To do.

Therefore, recently, a visual sensor (for example, CC
An automatic assembling apparatus having a D camera) has been proposed. As an example of such an automatic assembling apparatus, for example, JP-A-60
-56884, JP-A-60-123974 and JP-A-62-140724. In addition, Japanese Patent Application No. 1-71895, 2-
There is 55032. In these systems, an image of a work is taken in by a visual sensor, the work position is converted into a robot coordinate system by image processing from this image, and an assembly work is performed based on the work position in the robot coordinate system.

An automatic assembling apparatus having such a conventional visual sensor has a robot controller for controlling the operation of a robot hand and the like, and an image processing device for inputting and processing an image of a CCD camera, and further, These are for example RS23
It is usual to prepare an interface and a communication program that can be electrically connected to each other by a cable such as 2C and can communicate with both control devices. Then, a series of robot operation programs are input in advance into the robot controller. This program uses commands for sending and receiving information to the image processing apparatus through the interface and cable. Similarly, in the image processing device, a series of programs for performing recognition processing from the image input of the work and commands for exchanging information through the communication means are prepared in the series of programs. To configure.

[0005]

However, in the above-mentioned conventional technique or the proposed technique, an image processing device which can operate independently and a robot control device are prepared, and they are connected by a communication means to perform a series of assembling operations. There were the following drawbacks in order to make it possible. (1): Since the characteristic parameters used for pattern recognition of the work need to be stored in the image processing apparatus, and the programs and parameters used for robot operation and the like need to be stored in the robot control apparatus, the image processing apparatus and Each of the robot control devices has a keyboard or C that is an input / output device.
It is necessary to install a RT and a floppy disk drive as a storage device, which results in cost increase. (2): It is necessary to prepare a program that handles input / output in both of the above-mentioned devices, and development cost UP and lengthening cannot be avoided. (3): When the image processing device is replaced, each of the above parameters needs to be input again.

The present invention has been made in view of the above problems of the prior art, and an object thereof is to automatically grasp a work to be grasped by image processing and grasp the work based on the recognition. A control device for an automatic gripping device that uses vision and is capable of centrally managing parameters required for gripping operations and image processing.

[0007]

The control device of the present invention for achieving the above object includes an image processing unit for picking up an image of a work and calculating position information of the work from the image.
A control device for controlling an automatic gripping device comprising a gripping part for gripping the work on the basis of the position information, and a series of image processing commands for imaging the work and calculating position information from an image of the work, A program including a series of gripping operation commands for gripping a workpiece based on the position information, a storage unit that stores parameters for calculating the position information of the workpiece, the image processing command and the gripping operation in the program And an interpretation unit for identifying and interpreting the command, passing the image processing command to the image processing unit, and passing the gripping operation command to the gripping unit, and the parameter prior to execution of the program. And a transfer means for transferring to the image processing unit or the gripping unit.

[0008]

The parameter is stored in the same storage unit as the program including the image processing command and the gripping operation command, and is transferred from this storage unit to the gripping unit and / or the image processing unit in which it is used, if necessary. It

[0009]

Embodiments of the present invention will be described in detail below with reference to the accompanying drawings. In this embodiment, the present invention is applied to a control device of a so-called shuttle type automatic assembly device. <Overall Configuration> FIG. 1 is a perspective view of a system including a robot assembling apparatus and its controller according to this embodiment. This system is mainly composed of component supply devices 11 and 12, a shuttle type robot 6 for assembling, and an automatic assembly control device 17.

Reference numeral 9 is a shuttle in which the robot 6 is mounted. The shuttle 9 can be moved in one direction on the guides 16a and 16b by a shuttle moving motor 6e. Reference numeral 15 denotes a shuttle base, which is fixed to the floor in the present embodiment, and has guide rails 16a and 16b mounted on the upper surface in parallel with bolts or the like.

The shuttle 9 is controlled by the control device 17 so as to stop moving to a position taught in advance.
Further, the shuttle base 15 is provided with a parts supply device 11 for supplying articles for assembly stored in a pallet or the like.
12 is attached. In the system of this embodiment,
Two first component supply devices 11 and two second component supply devices 12 capable of supplying two different components are provided. Each of the supply devices can store a plurality of pallets 13a and 13b, and when the respective supply control units 14 and 21 empty the parts in the pallet, the empty pallets are exchanged for new pallets containing the parts. Supply device 1
1 and 12 are controlled.

Although the assembly robot 6 is fixed on the shuttle 9, the assembly robot 6 is provided along the guide rails 16a and 16b to the positions of the pallets 13a and 13b so that the parts can be taken out from the parts feeders 11 and 12. Moving.

The robot of this embodiment has what is called a horizontal multi-joint having four degrees of freedom. Two horizontal arms are driven by motors 6a and 6b, and the tip of the arm is raised and lowered by a motor 6c. The tip of the arm is rotated by 6d. A hand 7 is attached to the wrist of the robot 6, and the three motors 22a, 22
Fingers are driven by b and 22c, and a component can be gripped.

The above 6a, 6b, 6c, 6d, 6
The motors e, 22a, 22b, 22c operate according to the operation stored in advance by the controller 17.

The position of the robot 6 in FIG. 1 is at a position where the component on the pallet 13a on the component supply device 11 can be grasped.

The attaching / detaching portion 7a of the hand is detached by releasing a clamper (not shown) in response to a command from the control device 17. The hand holding table 10 is for holding a plurality of hands. The robot 6 can automatically attach an arbitrary hand on the holding base 10 in response to a command from the control device 17, and can remove the hand currently attached to the wrist from the hand holding base 10 and place it on the hand holding base 10. Is. Further, the assembling stage 8 is a place for assembling the parts grasped by the robot 6, and it is possible to provide a guide and an auxiliary function if necessary. When the operation of this auxiliary function is required, it can be controlled by placing a drive source on the assembly stage 8 and connecting the drive source to the control device 17 with a cable. Also, since the assembly stage 8 is mounted on the shuttle 9 as well as the robot 6, the robot 6 grabs parts from the pallet and slightly moves up. It is possible to continue the assembly operation.

Reference numeral 5 is a visual column, the root of which is a shuttle 9
It is fixed on. An XY table 3 is provided at the tip. The lens 1 is attached to the fixed portion of the XY table 3. The CCD camera 2 can be moved by stepping motors 4a and 4b in the XY2 directions. That is, the lens 1 covers the entire area of one pallet, but by moving the CCD camera 2,
It is possible to accurately capture an arbitrary partial area of the palette as image information. In other words, the entire surface of the pallet of 13b is moved through the lens 1 to a CCD which is sufficiently smaller than the image forming area, and the image information of only the place where the pallet of 13b is desired to be input is fetched.

Since the visual column 5 is mounted on the shuttle 9, it is moved together with the shuttle to the supply device that houses the parts to be assembled. The image of the pallet containing the parts to be picked up can be obtained by moving the CCD camera 2 (that is, the shuttle 9) to a position where the image can be input. Accordingly, the image-processed component can be gripped by the robot.

The automatic assembly control device 17 for controlling the entire assembling device uses a cable 18 to connect the robot 6, the shuttle 9, and the X.
It is possible to control an actuator such as the Y table 3 and the hand 7 and input a sensor (not shown). 19 is a CRT, and 20 is a keyboard for input / output. <Characteristics of Device of Embodiment> The functions peculiar to the system of this embodiment are mainly realized by the control device 17 described later. Such functions include :: In order to identify and recognize a component to be assembled (holding target) in the pallet, an image of the component in the pallet is captured and pattern matching is performed. With this recognition, no matter how the parts are stored in the pallet,
The parts can be recognized. : When the existence of the target part can be recognized in the pallet by pattern matching, the position of the part is recognized in the coordinate system of the visual system and further converted into the coordinate system of the robot system. The robot can grip the target component at the grip position in the coordinate system of the robot system. : An image processing program that describes a process for recognizing a target part and an assembly program that describes a control for gripping and moving the recognized part are stored in the RAM 52 in the command interpreting unit 87 described later. These two programs are executed by multitask control. : The image processing program (task) is activated by a special task activation instruction included in the assembly program. The activated image processing program is processed in parallel with the assembly program. : The image processing program and the assembling program are interpreted by the same instruction interpreting unit, and various instructions contained therein are converted into instruction codes. By converting into the instruction code, the difference in the property between the instruction in the image processing program and that of the assembling program is absorbed, and the unified management of the image processing program and the assembling program becomes easy. : The instruction code of the image processing program is stored in the dual port RAM described later. The instruction code of the assembly program is stored in the shared memory 60 described later.
By using the dual port RAM, it becomes possible to select the microcomputer of the image processing unit and the microcomputer of the operation control unit from those suitable for each purpose.

The structure of the control device 17 and the operation thereof will be described below. <Configuration of Control Device 17> FIG. 2 is an overall configuration diagram of the control device 17.

The controller 17 includes a robot operation controller 62 for driving and controlling the four-axis motors (6a to 6d) of the robot 6 and the motor 6e of the shuttle 9, and the three-axis motors (22a to 22c) of the hand 7. And a hand operation control unit 63 for driving and controlling the XY table drive motors 4a and 4b.
Image processing unit 64 for controlling the camera 2 and the camera 2.
And an interface 54 that controls the interfaces with the control units 14 and 21 of the two supply devices 11 and 12, an instruction interpreting unit 87 that interprets and executes a command of a robot program, the robot operation control unit 62, and a hand operation. The shared memory 60 is shared among the control unit 63, the image processing unit 64, and the instruction interpretation unit 87.

These components are connected to the system bus 59, and in the present embodiment, the bus 59 is a multibus. The instruction interpretation unit 87 will be described with reference to FIG.

Reference numeral 50 is a CPU and 51 is a ROM. As shown in FIG. 5, the ROM 51 includes a multitasking program for interpreting a robot language command interpreting program, and a visual processing command and a robot and hand motion command in the command interpreting program as required. An operating system (hereinafter abbreviated as "OS") and various programs supporting them are stored in advance. Details of these programs will be described later.

The RAM 52 of the instruction interpreting unit 87 stores the image processing program and the operation program for the robot / hand, which are input by the keyboard 20.

The elements in the instruction interpreting section 87 are connected by a local bus 86. That is, the local bus 86 has a ROM 51, a RAM 52, and an interface IF 53.
Are combined. The data in RAM52 is CRT1
9 can be displayed, and can be input with the keyboard 20 or deleted. The keyboard 20 is used to issue an instruction for the actual operation of this system.

Reference numerals 62 and 63 are operation control units for the robot and the hand. When the instruction code corresponding to the robot operation and the hand operation is written in the shared memory 60, the control units 62 and 63 perform the operation processing according to the instruction code, and each device, that is, each motor 6a to 6e.
Rotation and stop. As described above, the instruction code is written in the shared memory 60 by the instruction interpretation unit 87.

The image processing unit 64 is a portion for performing image processing for recognizing the parts, and performs processing by the instruction code written in the IF 61.

Reference numeral 69 denotes a known magnetic external storage means, for example, a floppy disk drive or a hard disk drive, which stores a processing instruction of a robot hand operation instruction image by operating the keyboard 20 through the IF 53. It is possible to make it.

Further, by bringing a floppy disk stored in an external computer or the like or a floppy disk already stored in the RAM 52, processing instructions and operation instructions are stored in respective designated areas in the RAM 52. It is also possible to do so.

The control units 14 and 21 of the component supply device have optical communication circuits 56a and 56b, and can exchange information with the optical communication circuits 55a and 55b on the shuttle 9. Each of these optical communication circuits has a light emitting means and a light receiving means, and the light emission and the light reception face each other, and when the robot 6 is stopped at a predetermined position of the supply device, communication is possible.

The interface IF57 is an interface for communicating with an external computer or the like,
The information in this assembly device is sent to the outside and the information is input from the outside. For example, it is possible to calculate the timing of automatic delivery of pallets by communicating with a computer that manages an unmanned warehouse.

Reference numeral 59 denotes a common bus connecting the IF 61 of the robot operation unit 63 and the hand operation unit 61 of the CPUs 50 and 62 and the shared memory 60, which is a multi-bus in this embodiment.

The instruction code generated by being interpreted and executed by the CPU 50 of the interpretation unit 87 is set in the shared memory 60.
The robot operation controller 62 or the hand operation controller 63 receives these instruction codes. When the control units 62 and 63 perform the actual operation and the operation ends, the shared memory 60
The control unit writes an operation end signal to the top.

When the instruction interpreted and executed by the CPU 50 is an instruction related to image processing, the instruction code is a two-port RAM (2-PORT-RA) provided in the IF 61.
M), and when the processing result communication is required when the image processing is completed, the processing result and the processing end signal are also stored on the two-port RAM in the IF61 as in the case of the assembly operation command described above. Write in. Image Processing Unit 64 FIG. 3 is a detailed block diagram of the image processing unit 64.

Reference numeral 65 is a CP for controlling the entire image processing section.
U. In the two-port RAM 61, the controller 1
The bus 59 shown in FIG. 2 and the bus 72 shown in FIG. 3 are connected from both directions so that data can be exchanged with the other elements, and reading and writing can be performed from both directions.

Two buses are connected between the CPU 50 and the CPU 65.
By providing the port RAM 61, the CPU 50 and C
It is possible to change the type of CPU of the PU 65. It should be noted that there is no problem even if the CPU 65 and the CPU 50 have the same format as the communicable memory 60 shown in FIG. In addition, the robot operation control unit 6 of FIG.
2. The data transfer between the hand operation control unit 63 and the command interpretation unit 87 is also performed by the two-port RA as in the image processing unit 64.
There is no problem even if communication is performed by providing M61 as an interface.

In the image processing unit 64, reference numeral 2 is a camera for capturing an image, and in the present embodiment, the lens is fixed to the XY table. XY table 3 is C
By moving the CD within the imaging plane, it is possible to capture the image information of the place where the pallet is required. The image signal captured by the camera 2 is digitized by the A / D conversion unit 73 at the brightness level and stored in the frame memory 75.

FIG. 4 shows the appearance of a pallet used for storing components in the component supply device. As shown in the figure,
In the pallet, parts are contained in divided cells. The part is specified by the cell position information of the pallet. Since the assembly apparatus recognizes the accurate position of the component by image processing, it is not necessary to fix the component in the pallet accurately in advance. Also, the cells of the pallet do not have to have exact dimensions. It suffices to know the approximate location of the cell containing the required component. That is, if the cell position of the required component is specified in the robot program, image processing is performed on the image around the cell position, and the accurate position of the required component is found in the cell by pattern recognition.

If the field of view of the CCD camera 2 is set so as to cover the entire pallet, the cost is increased and the CCD element is made large, or the resolution is deteriorated and the small CCD element is accepted. Must ensure that the entire pallet is in view. Therefore, in the present embodiment, the camera 2 is of a movable type so that only a necessary portion of the image of the palette is taken out. Therefore,
The camera 2 is provided with an XY table drive unit 76. The drive unit 76 rotationally drives the two stepping motors 4a and 4b of the XY table to move the camera to a predetermined position. The area setting unit 74 has a function of limiting the area when it is not necessary to capture image information of the entire area of the CCD when capturing an image with the camera 2. In the assembling machine, the shape and size of the assembled parts are known in advance, so if you want to capture only the image information of one part, program a processing command that limits the area within the image processing command. deep.

The binarizing unit 66 uses the image information stored in the frame memory 75 as a reference based on the brightness level designated in advance by a processing command, for example, a bright one is "1" and a dark one is "0". It is binarized as ". The binary image is stored in the frame memory 75 again.

The labeling section 67 performs labeling.
In this labeling process, for example, a group of connected dot clusters, that is, a group of "1" clusters is found in the binarized image information, and a different label is attached to each group.

The feature parameter calculator 68 calculates, for example, the center of gravity position area and the principal axis of inertia for each of the labeled connected regions. Whether the image is the image of the designated part by comparing the characteristic parameter of the image of a certain work calculated by the calculation unit 68 with the reference characteristic parameter stored in the two-port RAM 61. To determine. When it is determined that the part of the image is the specified part, the position and the tilt of the part in the visual coordinate system are obtained, and the two-port RA is calculated.
Convert to the robot coordinate system using the conversion parameters to the robot coordinate system stored in M61 beforehand,
The data is stored on the two-port RAM 61 as position data. If the position data is required in the robot operation command, the position data is read, and the operation command that operates based on the result is executed to grasp the part in the pallet with the position information calculated by the image processing unit and assemble it. Can be done. Further, the characteristic parameters and the coordinate conversion parameters stored in the two-port RAM 61 are stored in the RAM 52 of FIG. 2 where the battery is backed up and are automatically set when the power of the control device is turned on. It is also configured to be stored in the two-port RAM 61.

In FIG. 3, the ROM 71 stores in advance a program for performing a series of image processing. <Program Structure> FIG. 5 shows the storage status of the program stored in the ROM 51 for the local memory of the instruction interpretation unit 87 described with reference to FIG.

The ROM 51 has a start program 100 that is executed first when the power is turned on, a multitask OS 101 that manages and executes a plurality of tasks, a text editor 102 for creating a program, and a created text. A compiler 103 for converting a program into an interpretable intermediate code, and interprets the intermediate code converted by the compiler 103, and a lower control unit (robot motion control unit 62, hand motion control unit 63, image processing unit 64) Instruction program 104 for generating an instruction code (FIGS. 13 to 15) for a keyboard, a keyboard, and a CR
An IF driver program 105 for controlling the input / output of an input / output device such as T, a robot task display program 106 for displaying the operation state of the robot, the state of the robot task such as the created assembly operation program, and an image processing unit. A 64 task status display program 107 for displaying the status of the task, such as the created task operation program, is stored.

FIG. 6 is a diagram showing a data storage state in the local memory RAM 52 of the instruction interpreting section 87. In the RAM 52, an assembling operation program 108 including a series of operation instructions for robot assembling operation.
(FIGS. 9 and 10), a vision operation program 109 (FIG. 11) consisting of a series of processing operation instructions for the processing operation of the image processing unit, the target position data 110 of the XY stage, and the shuttle. Target position data 111, target position data 112 for positioning the robot, and reference feature parameters 113 indicating the features of the work to be assembled.
And the coordinate conversion parameter 114 are stored. This parameter 114 is a calculation parameter used for coordinate conversion calculation from the robot coordinate system to the XY stage coordinate system, and XY
It is the same calculation parameter from the stage coordinate system to the robot coordinate system.

The RAM 52 is volatile, but the data stored in it is important, so it is backed up and backed up as will be described later. <Robot Program> FIGS. 7 and 8 are views showing a process of an assembling operation for assembling a work by receiving two component supplies from the component supply units 11 and 12 in the assembly system of FIG. That is, FIG. 7 shows a state in which the shuttle 9 is stopped at a position where the robot 6 can access the component A stored in the pallet 13a of the first component supply unit 11. FIG. 8 shows that the shuttle 9 is placed at a position where the robot 6 can access the component B (the component B is stored in the pallet 13b of the component supply unit 12) which is held after the assembly of the component A. Shows that it has stopped.

As can be seen from FIG. 7, particularly FIG. 8,
While the robot 6 is performing the operation of assembling the part A, as long as the camera 2 is above the pallet 13b storing the work B, the assembling operation of the part A and the imaging operation of the part B are performed. Can be done in parallel. Due to the parallelism of this assembling operation processing and the imaging operation (image processing),
As shown in FIG. 6, it is significant to convert the assembly operation program and the vision operation program into tasks.

FIG. 9 is a series of robot operation command groups (program) for assembling the part A, FIG. 10 is a series of robot operation command groups (program) for assembling the part B, and FIG. 11 is for the part B. It is a series of image processing command groups (programs) for performing image processing on an image and calculating position information.

Note that FIGS. 9 to 11 do not show a program for image-processing the image of the part A to calculate the position information. Further, in order to complete the work, for example, if the part C is required, a series of image processing command groups (programs) for image-processing the image of the part C to calculate position information, A series of robot operation command groups (programs) for assembling the part C are required in addition to the programs shown in FIGS. 9 to 11. Also,
For the sake of convenience, the program of FIG.
The program shown in the figure is called program 2, and the program shown in FIG. 11 is called program 3.

The programs shown in FIGS. 9 to 11 are shown in FIG.
The operation of FIGS. 8 and 9 is described. In order to assemble the part A and then assemble the part B, the program 1 (FIG. 9) must first be started in advance. Program 1 is running and program 1
The program itself is activated. The program 2 is activated by the program 1 by calling CALL 2 on line 390 and calling the program 2. In order to start the assembly operation of the component C after the assembly of the component B is completed, the CALL 3 of the line number 250 is executed and the program 3 (not shown) is called. If the image of the part B and its image processing are possible during the assembly operation of the part A, the line number 3
At 60, execute START 3,2 to call program 3 (FIG. 11) under multitasking control. That is, the program 1 and the program 3 operate in parallel. During the assembly operation of the part B, the part C
When it becomes possible to obtain the image of
At line number 220, START 3,3 is executed to operate the programs 2 and 3 in parallel. The second operand of the START instruction that is the parallel processing start instruction is the procedure number of the start target,
The third operand represents the procedure number in the starting program. Although the third procedure of the program 3 is not shown, since the part C does not have a special shape, that is, it does not require special image processing according to the shape, the third procedure of FIG. The same program can be used.

The meaning of the instruction of each statement in the programs of FIGS. 9 to 11 will be described later. <Parallel Processing> As shown in FIGS. 7 and 8, in the robot system of this embodiment, the assembly operation and the image processing can be performed in parallel. Further, as shown in FIGS. 9 to 11, the programs describing these operations are also conscious of parallel processing. Therefore, the characteristic control of the control device 17 in order to realize the parallel processing will be described below. These features are, as shown in FIG. 12 ,: The so-called robot program (the programs of FIGS. 9 to 11) is interpreted by the instruction interpretation program 104 (FIG. 5) of the instruction interpretation unit 87. The program 104 executes commands (statements) in the program relating to an assembly operation (robot operation control unit 62 or hand operation control unit 63) or image processing (for the image processing unit 64). And the corresponding instruction code is generated. : The instruction code generated from the instruction related to the assembling operation is written in the shared memory 60, and the instruction code generated from the instruction related to the image processing is written in the two-port RAM 61. : Robot operation controller 62 or hand operation controller 6
3 executes the instruction code written in the shared memory 60. Further, the CPU 65 of the image processing unit 64 executes the instruction code written in the two-port RAM 61.

FIGS. 13 to 15 are diagrams showing an example of the memory storage format of the instruction code. Instruction Code FIG. 13 is a diagram showing a general storage format of the instruction code. The storage format of FIG. 13 is "next instruction storage address pointer" 200 indicating the memory address where the next instruction is stored.
And "P code" 201 indicating the type of instruction, and the type of parameter for supplementing the instruction (for example, whether it is a direct number, a variable, a real number, or an integer) , A data portion 203A of the parameter thereof, a delimiter 204A indicating the end of the data, and a checksum 205 for indicating the end of the instruction code and detecting an error in the contents are stored in this order.

Some instructions do not require parameters in the instruction code. In the instruction code of such an instruction, as shown in FIG. 14, a checksum 205 indicating the end of the instruction is stored after the P code 201.

In the case of an instruction that requires a plurality of parameters in the instruction code, as shown in FIG. 15, after the P code 201, the data type 202 and the data 20
3, the checksum 205 is stored after the required number of sets of the delimiters 204 are stored.

In this manner, the instruction statement is reconfigured into an instruction code (intermediate code) consisting of the instruction type (P code 201) and the parameter and stored in the memory, so that the assembling operation program 108 and the vision are executed. The operation program 109 can be stored in the same format.
Further, the fact that the instructions of the assembling operation program 108 and the vision operation program 109 are both converted into the same type of instruction code means that the memory for storing them can be shared. That is, specifically, instead of the two-port RAM 61 which is a memory for storing the instruction code for the image processing unit 64, the instruction code for the image processing unit can be stored in the shared memory 60. Means there is. In the present embodiment, the reason why the two-port RAM 61 is used in addition to the shared memory 60 is to avoid access competition to the memory due to the unification of the memory, and to use the two-port RAM 61 in two different directions. High-speed processing is possible by being simultaneously accessed from each other, and the assembling operation and the image processing can be easily parallelized, and further, the image processing unit 6
The CPU of No. 4 has a different architecture from the CPU of the instruction interpreting unit 87. <Control for Parallel Processing> FIG. 16 is a diagram illustrating a structure between programs for controlling parallel processing in the present system. A series of procedures for interpreting the assembly operation program 108 is called a "robot task", and starting interpretation of the assembly operation program 108 is called "activating a robot task". Similarly, a series of procedures for interpreting the vision operation program 109 is called a "vision task", and starting interpretation of the vision operation program 109 is called "activating a vision task". The activation of the vision task is performed by the above-mentioned START command. In this assembling system, since the procedure of auxiliary image processing is performed during the assembling process, the START statement for activating the vision task is described in the assembling program. This facilitates competitive arbitration between robot tasks and vision tasks. Further, the robot task is activated when the start key is input from the keyboard 20 of the control device 17.

The robot command interpreted in the robot task is converted into a command code (FIGS. 13 to 15) and stored in the shared memory 60 as described above.
In addition, the image processing command interpreted in the vision task is also converted into a command code in the same manner, and the two-port R
It is stored in the AM 61. The instruction code of the shared memory 60 is fetched and executed by the robot operation control unit 62 and the hand operation control unit 63. Robot operation control unit 6
2. Although not shown, the hand operation control unit 63 has a dedicated microprocessor, a memory for storing the programs shown in FIGS. 18 and 19, a servo control circuit, a servo amplifier, etc., respectively. There is.

As shown in FIG. 16, the instruction interpreting program has three flags (MWF, TWF, PWF). That is, "operation completion wait flag" (MW
F) indicates that among the instructions of the robot task, an instruction that needs to wait for the completion of the execution of the instruction is currently being executed by the robot task. The operation completion wait flag MWF is set by executing an instruction such as MOV P (4). These instructions are
This is because it is an instruction that requires an action such as actual movement of the robot, and it is necessary to wait for the end of that instruction before executing the next instruction. For control commands such as IF ... THEN ..., since the actual movement of the robot is not involved, the operation completion waiting flag MWF is not set.

Further, "process end wait flag" (PWF)
Indicates that among the instructions of the vision task, an instruction that needs to wait for the end of execution of the instruction is currently being executed by the vision task.

These MWF and PWF are for preventing two or more instructions from being overlapped and executed in the robot task or the vision task.

The "timing wait flag" TWF was introduced into this system for the following reasons. That is, under multitask control, the instructions in the robot task and the instructions in the vision task are executed in parallel. Then, in order to start one command with the robot task, it may be necessary to confirm the end of one command with the vision task, that is, to arbitrate between the two tasks. For example, when the CCD camera 2 is capturing an image of an object,
This is because it is necessary to prevent the imaging of the robot 6 from becoming impossible due to the arm of the robot 6 entering the field of view of the camera 2 or the movement of the shuttle 9. The above MWF and PWF are
To the end, it arbitrates instruction execution between the same tasks. Therefore, the "timing wait flag" T is used as a flag for one task to suspend the other task.
There is WF.

In the system of this embodiment, a WAIT instruction is prepared as an instruction for one task to temporarily suspend the other task. This WAIT instruction is mainly used in the vision task, and when it is executed, the stop flag S
F (suspended flag) is set. The flag SF is provided in the two-port RAM 61, and the WAIT instruction is automatically added by the system (compiler 103 in FIG. 5) without explicitly describing those instructions in the program. There is. That is, in the example of FIG.
The compiler 103 automatically adds the WAIT command after the imaging command (GET) for imaging with the CCD camera 2 or after the pattern recognition command (MATCH). In other words, when the GET instruction or the MATCH instruction is executed, the WAIT instruction is automatically executed and the flag SF is set. The flag SF is reset at the end of the special instruction (GET instruction or MATCH instruction). Therefore, on the robot task side, a command for inspecting the set state of the flag SF, that is, a timing confirmation command (for example, a CHECKF command) is inserted into the robot task to coordinate the robot task and the vision task.

Further, the instruction interpretation program is provided with two counters (RPC and IPC) as shown in FIG. The counter RPC stores the line number of the next execution instruction in the robot task assembly operation program 108. For example, if the next instruction to be executed by the robot task is SSHHUTL at line number 300 (FIG. 9), 300 is stored in the counter RPC. Further, the counter IPC is the vision operation program 10 of the vision task.
The line number of the next execution instruction in 9 is stored. For example, the command to be executed next by the vision task is BIN at line 20.
If it is (FIG. 11), 20 is stored in the counter RPC.

According to FIGS. 17, 18 and 19,
The parallel execution control of the robot task and the vision task is explained. When the start of the assembling operation is instructed from the keyboard 20 in the control step S200 of the robot task , the command interpretation unit 87 causes the step S201.
Then, the robot task is activated to start the interpretation of the assembly operation program 108 in the RAM 52 (step S201).

At step 202, it is checked whether the robot task is activated. The reason why the robot task is activated in step S202 is that a statement for stopping the robot task can be described in the robot program itself or in the vision task program itself. .. Therefore, when the robot task is stopped, only the vision task is executed, and in order to programmatically restart the robot task, it is necessary to execute an instruction to start the robot task in the vision task. 9 to 1
When the program in the example of FIG. 1 is being executed, the robot task is not stopped programmatically, so the routine proceeds to step S203, where the judgment of the operation completion waiting state is made based on the operation completion waiting flag MWF. As described above, since the MWF is set for a move instruction or the like, if such an instruction has not been executed in the previous cycle, or an unnecessary instruction of the MWF set has been executed. In this case, since MWF = 0, the process proceeds to step S204.

When MWF = 0, the process proceeds to step S204, one instruction is read from the assembling operation program 108, and the instruction is interpreted. When one instruction is interpreted, the counter RPC is changed to indicate the next line number.

In step S205, the contents of the interpreted instruction are the business task start instruction ("START").
3 "). If it is a vision task activation instruction, the vision task activation processing is performed in step S220, and the flow advances to step S221.
If the command is not a task start command, step S20
Proceed to step 6 and confirm the timing (command CHECKF
Etc.) is determined.

If it is determined in step S206 that the instruction content is not the timing confirmation instruction, the robot operation unit 6
2, an instruction that needs to issue an instruction code to the hand operation unit 63 and the image processing unit 64 (for example, a movement instruction of the shuttle 9 or an operation instruction of the hand 7) or an instruction that does not need to issue an instruction code (for example, IF It is determined in step S207 whether it is a control command such as "THEN ...".

If the instruction code needs to be issued, in step S208, the instruction code is determined according to whether the instruction code is a robot motion related instruction or a hand motion related instruction. , And is stored in the shared memory 60 so that the robot operation control unit or the hand operation control unit can be executed separately. The robot motion control unit or the hand motion control unit is shown in FIG.
According to the control procedure of FIG. 9, the corresponding instruction code is fetched from the shared memory 60 and executed. Then, in step S209, the operation completion waiting flag MWF is set.
Then, the process proceeds to step S221.

For an instruction that needs to issue an instruction code to the operation control units 62 and 63, it takes a long time to complete an operation corresponding to the instruction, such as movement of the shuttle 9 (MOV) or movement of the robot 6. It is a thing. The flag MWF is set in step S209 in order to prevent the movement command and the like from being continuously issued. However, instructions that do not require issuance of instruction codes (for example, control instructions such as IF ... THEN ...) should be continuously executed. When it is determined in step S207 that such an issue-unnecessary instruction is to be executed, the process proceeds directly from step S207 to step S208, and the instruction is directly executed without setting the MWF. Then, the process proceeds to step S221.

If it is determined in step S206 that the instruction content is the timing confirmation instruction, step S21
Go to 0 and flag SF in the two-port RAM 61
Check the set state of. As described above, this flag SF is set by another task (in this embodiment, a vision task) when a special instruction (GET or MATCH) is executed by that task, and at the end of that instruction, the task is set. Resets. Since the flag SF is placed in the two-port RAM 61, it can be accessed not only by the image processing section 64 but also by the instruction interpreting section 87. Therefore, if it is determined in step S210 that the flag SF has been set, the interpretation / execution of the instruction in the robot task must be interrupted. Therefore, in order to store that fact, the flag TWF is set in step S212. To set. Then, in step S209, the flag MWF is set. That is, the flag MWF is set both when the interpreting unit 87 interprets an instruction that requires the issuance of an instruction code and when the TWF is set by interpreting the timing confirmation instruction.

In step S221, it is checked whether or not the vision task is activated. If not activated, the process returns to step S202.

When a certain robot program is being executed, the flag MWF is not set as long as an instruction that does not need to be issued an instruction is executed. Therefore, the process proceeds to step S202 → step S203 → step S204. Interpret and execute the next instruction. That is, the instructions of the robot program are continuously executed without waiting.

On the other hand, the robot operation command in the line numbers 300 to 350 of the program shown in FIG. 9 is the flag MWF.
Set. In this case, the control is step S20.
The process proceeds from step 2 to step S203, and since the flag MWF has already been set, the process proceeds to step S214. In step S214, in order to check whether the flag MWF is set by the execution of an instruction that needs to issue an instruction code or by the execution of a timing confirmation instruction,
Check the set state of the flag TWF.

If the TWF flag is reset in step S214, it means that the flag MWF has been set by the execution of an instruction that requires the issuing of an instruction code, so the flow advances to step S215 to check the completion of the robot operation. .. This operation completion check can be made by checking whether or not the instruction code remains in the shared RAM 60. Shared memory 60
The instruction code is stored in the instruction interpreting program in step S208. The instruction code in the shared memory 60 is cleared by the robot operation control unit 62 and the hand operation control unit 63.

FIG. 18 is a flow chart showing a control procedure in the robot operation control section 62 and the hand operation control section 63.

In step S300, the robot operation control section 62 and the hand operation control section 63 respectively scan the shared memory 60 and check whether the instruction code for themselves is stored in the memory 60. Whether or not the instruction code is for oneself is determined by the P code (first
It is determined by the type of instruction shown in FIGS. 3 to 15). If the instruction code for itself is found, the instruction code is executed in step S301, and is ended in step S302 (for example, the end of the raising or lowering operation of the arm).
Wait for When the operation is completed, the instruction code is cleared in step S303.

Also in the control procedure of the instruction code processing in the image processing section 64 of FIG. 19, the scanning / execution / clearing of the instruction code is the same as that of FIG.

Returning to the flowchart of FIG. 17, if it is confirmed in step S215 that the instruction code stored in the shared RAM has been cleared, step S216.
Then, the flag MWF is reset. Then, in step S221, if the vision task is not activated, the process proceeds to step S221 → step S202 → step S203 → step S204 to execute the next command in the robot program.

In the process for determining the reason for waiting in step S214, when it is determined that TWF = 1, in order to wait for the flag SF to be reset by the vision task,
The flag SF is checked in step S217. This flag S
The fact that F is set (SF = 1) means that we still have to wait.
The process returns to step S202 via 221.

If it is confirmed in step S217 that the flag SF has been reset, step S218 is performed.
The flag TWF is reset with to indicate that the waiting state has been released. Further, step S212 → step S20
In step S21, the flag MWF set in the process of 9 is set.
Reset with 6. Parallel execution of robot task and vision task As described above, the instruction code issuance is executed, the unnecessary instruction is executed, and the timing confirmation instruction is executed. Next, the case where the vision task is activated will be described in more detail.

When the START 3 instruction at line number 360 is executed in step S205, the vision task is activated in step S220, and then the process proceeds to step S221. In step S221, it is determined based on the flag PWF whether the present state is in the waiting state waiting for the completion of the processing of some instruction of the previously executed vision task. This flag PWF has the same function as the flag MWF in the robot task. If no image processing command has been issued before, PWF = 0, so step S223
Proceed to and interpret one image processing instruction according to the counter IPC. Step S223 and Step S224 are
The type of instruction interpreted in step S223 is determined. That is, if the command is the END command (statement at line number 50 in FIG. 11), the vision task is stopped at step S225. If it is not the END instruction, it is checked in step S226 whether or not the instruction requires the issuance of an instruction code. The instruction code needs to be issued is an instruction that requires some processing in the image processing unit 64, such as an imaging instruction (GET), a binarization instruction (BIN), and a matching instruction (MATCH). is there. Further, control instructions such as IF ... THEN ... and the above-mentioned WAIT instruction only need to access the two-port RAM 61, and the processing by the image processing unit 64 is unnecessary, so that the issue of the instruction code is not necessary.

When executing an instruction that requires the issuance of an instruction code, in step S227, the two-port RAM is used.
The instruction code is written in 61. And step S2
In step 28, the processing completion waiting flag PWF is set, and the process returns to step S202.

Prior to the interpretation of the vision task commands,
If the previously executed instruction has not been processed yet, the process proceeds from step S222 to step S230. In step S230, it is checked whether the previously issued instruction code in the two-port RAM 61 has been cleared by the image processing unit 64. This instruction code is cleared in 19th
The image processing unit 64 performs this according to the control procedure shown.

If it is determined in step S222 that the instruction code has already been cleared, step S231.
The flag PWF is reset with.

As described above, the parallel processing of the robot task and the vision task is performed.

As described above, when the interpreting unit 87 interprets and executes a special command (GET, WATCH) of the vision task and stores the command code in the RAM 61, according to the flow chart of FIG. , Image processing unit 6
4 fetches the instruction code from the RAM 61 and executes it in step S312. And step S3
At 11, the flag SF is set in the RAM 61. When the execution of such a special instruction is completed, the operation of clearing the instruction code and the operation of resetting the flag SF are performed together in step S313.

FIG. 20 shows the concrete assembly program shown in FIGS. 9 and 10 and the vision operation program shown in FIG. 11 interpreted by the instruction interpretation program shown in FIG. The operation command 62
The timing chart shows how the robot unit and the image processing unit 64 operate when the image processing unit 63 and the image processing unit 64 execute the control procedures of FIGS. 18 and 19. That is, FIG. 20 shows part of the assembling operation of the part A, the assembling operation of the part B, and part of the assembling operation of the part C. Further, in FIG. 20, for convenience of explanation, at the time of starting the present time chart, the robot 6 grasps the component A with the hand 7 from the pallet 13a of the component supply unit 11, and the robot 6 is retracted above the pallet 13a. It is assumed that

The assembly operation sequence of this embodiment will be described below with reference to FIG.

During the periods T ₁ , T ₂ , T ₅ , and T ₆ , only the robot operation is controlled. That is, in the period T ₅ , the shuttle 9 starts moving to the position where the part B is assembled (line number 300).
Of SSHUTL instruction) and, in the period T _6, CSET command X-Y table 3 is then assembled movement start the visual object position of the pallet 13b of the part B (row number 310) to. Next, the robot 6 moves to the upper empty point of the assembling stage 8 while holding the part A in the hand 7 in the period T ₁ , and further stops descending vertically to the assembling stage 8 (line number 3).
MOV command of 20,330).

For T ₃ , T ₄ , T ₇ , T ₈ , T ₉ T ₁₀ ,
Robot control and image processing control are performed in parallel.
It is confirmed that the movement of the shuttle 9 and the movement of the XY stage 3 are completed (the lines 340 and 350, CHECK and C).
Then, the image processing control is started (36).
START command at line 0). As a result, the robot task and the vision task are executed in parallel.

In the control in the robot task in the parallel processing, the hand 7 is moved and the part A is released during the period T ₃ (OUT instruction at line number 370). And in the T ₄ period,
Robot 6 raises / stops hand 7 (MO on line 380
V command) to complete the assembly of the part A.

In the control on the image processing side which is carried out concurrently with the control in the robot task, the image of the CCD camera 2 of the part B to be visualized is image-processed by the image processing section 64 during the period T _7.
(The GET command of line number 10) is taken in, and the image is binarized by the image processing unit 64 in the period of T ₈ (command BIN of line number 20), and further binarized in the period of T _9. The image feature extraction process (command SOP of line number 30),
The position detection process (command MATCH of line number 40) of the visual target part B is performed in the period T ₁₀ .

At the time when the T ₄ period has expired and the T ₁₀ period has further expired, the hand 7 is capable of gripping the component B in the component supply unit above the assembly stage 8 and the image processing unit 64. Detects the position of the part B on the pallet 13b.

When the hand 7 retreats above the stage 8 by the MOV instruction at line number 380, program 2 (FIG. 10) is called at line number 390. This program 2 describes the procedure for receiving the component B from the supply unit 12 and assembling it.

At line number 100 of program 2, the instruction CHECKF 3,2 is executed. This command is a command for confirming the end of the task 3 (that is, the program 2 of the vision task, that is, the image processing of the part B). When it is confirmed by this command that the position detection processing of the component B has been completed, the processing of the line number 110 and thereafter is performed.

At T _{11 to} T ₁₆ , T ₁₉ and T ₂₀ in FIG. ₂₀ ,
Only robot control is performed. In the robot control, line number 110 reads the position detection result as the movement target position of the robot 6.

P (20) = VREAD (2) Then, in order to grip the component B, at T ₁₁ , the robot 6 moves the hand 7 to the upper point of the pallet 13b (line number 12).
0), and at T ₁₂ , descend and stop (line number 130). Next, at T ₁₃ , the hand 7 grips the target part B (OUT command of the line number 140), and at T ₁₄ , the robot 6 stops moving the hand 7 holding the part B to the upper point of the pallet 13b (line number). 150).

[0098] Operation of the robot control for component B of the _{T 15, T 16, T 19} , T 20 is, T ₁ for the component A,
It is the same as T ₂ , T ₅ , and T ₆ . Also, T ₁₇ , T ₁₈ , T
_{The operations in} which the robot control of the part B of _{21 to} T ₂₄ and the image processing control of the part C are performed in parallel are the same as the operations of T ₃ , T ₄ , and T _{7 to} T ₁₀ .

FIG. 21 shows line numbers 330 to 39 of FIG.
The instruction interpretation in the instruction interpreting unit 87, the robot operation control unit 62, the hand operation control unit 63, and the image processing unit 64 in the process of sequentially executing the instruction of 0 and the instructions of the line numbers 100 to 120 in FIG. Details of execution It is a time chart.

First, at T ₁₀₀ , the instruction interpreting section interprets the robot operation instruction at line number 330 in FIG. 9, generates an instruction code for moving the robot, and transfers the instruction code to the robot operating section 62. The instruction code is set in the shared memory 60 of FIG.

At T ₁₂₀ , the robot operation control unit 62 drives the robot 6 by the received instruction code through the shared memory 60 and stops the arm at the instructed position. The robot operation control unit 62 terminates the execution of the instruction code by clearing the instruction code, and then the instruction interpretation unit 8
Contact 7. At T ₁₂₀ , the instruction interpretation unit 87
Waits for the end of execution of the instruction code from the robot operation control unit 62.

At T ₁₀₁ , the instruction interpreting section 87 interprets the robot operation instruction at the next line number 340 in FIG. 9, generates the instruction code for shuttle stop confirmation (CHECK), and causes the robot operation control section 62 to Send the instruction code through the shared memory. The robot operation control unit 62 confirms the shutdown state of the shuttle by the instruction code, and notifies the instruction interpreting unit 87 of the termination of the instruction code execution via the shared memory 60. Subsequently, the instruction interpreting unit 87, which has been in the waiting state for resetting the MWF, restarts the interpretation of the next operation instruction.

T₁₀₂ Is the command interpreter 87
Interpret the bot operation command and confirm the stop of XY table 3
To the robot operation control unit 62.
T ₁₀₁ Instruction code delivery and instruction code
Hand over the code execution.

At T ₁₀₃ , the instruction interpretation unit 87 causes the line number 360 to be displayed.
The robot operation command (START) is interpreted, and the interpretation execution of the image processing command group (vision task) shown in FIG. 11 is started. After T ₁₀₃ , the robot operation instruction group and the image processing instruction group are interpreted in parallel.

At T ₁₁₀ , the instruction interpreting section 87 interprets the image processing instruction of line number 10 in FIG. 11, generates an instruction code for capturing the image of the CCD camera into the image processing section, and instructs the image processing section 64. The instruction code is sent via the two-port RAM 61.

At T ₁₃₀ , the image processing unit 64 which receives the image capture instruction code from the two-port RAM 61.
Captures the image of the CCD camera 2 in the image processing section, and after the completion of the capture, informs the instruction interpreting section 87 of the completion of execution of the instruction code via the two-port RAM 61.

At T ₁₀₄ , the instruction interpreting section 87 interprets the image processing instruction at T ₁₁₀ and completes the instruction code set.
The robot operation command at line number 370 in FIG. 12 is interpreted.
In T _{10 4,} and generates an instruction code to move the hand 7 and excisional the instruction code in the shared memory 60. In the next state, the instruction interpreting unit 87 waits for the completion of the execution of the instruction code issued to the image processing unit at T ₁₁₀ and waits for the completion of the execution of the instruction code issued to the robot operation control unit 62. I'm running.

At T ₁₁₁ , the end of execution of the instruction code is notified from the image processing unit 64 via the two-port RAM 61. Further, in the state where the end of execution has not been notified from the robot operation control section 62, the instruction interpreting section 87 interprets the image processing instruction of the line number 20 in FIG. 11 and generates the instruction code of the binarization processing of the captured image. And two port R
Set on AM61.

In the same manner, the instruction interpreting unit 87 interprets the finished instruction group at the time when any one of the robot operation instruction and the image processing instruction has been executed (T ₁₀₅ , T ₁₁₂ , T ₁₁₃ , T ₁₀₆ ).

Further, at T ₁₁₄ , the instruction interpretation unit 87 interprets the image processing instruction at line number 50. Since the command at the line number 50 is the interpretation execution end command (END), the command interpreting unit 87 stops the vision task.

After T ₁₀₇ , the interpreting unit 87 sets the interpreting instruction only for the robot operation instruction group. <Image Processing> Image processing performed by the image processing unit 64 will be described below.

The purpose of image processing in this system is to
The recognition of the product and the posture of the part in the pallet
Detection and detection of the barycentric coordinates of the component.Feature parameters FIG. 22 is a plan view of the part B as an example used in this embodiment.
It is a figure. Part B is a flat plate and has holes of three different sizes.
Is open. FIG. 23 shows the basis of the characteristic parameters of the part B.
It represents a quasi value. s₁ Is hole H₁ Area of s₂ Is a hole
H₂ Area of s ₃ Is hole H₃ Area.

In this embodiment, the hole H₁ Coordinates of the center of
(X₁ , Y₁) Is the origin of the part, and the robot 6
The image processing unit 64 positions the parts so that the parts can be gripped.
Use as a reference point when calculating the position. Also, the hole H₂ Heart of
(X₂ , Y₂) And hole H₁ Center of (X₁ , Y₁) And through, hole
H₂ From hole H₁ The vector α in the direction of
Think of it as an inclination. This reference axis α (vector
α) and hole H₁ Center of (X₁ , Y₁) And hole H₃ Center of (X
₃ , Y₃) Is the angle formed by the line segment connecting₃ And Also,
Hole H₁ Center of (X₁ , Y₁) And hole H₂ Center of (X₂ , Y₂)
The distance to₂ And l₂ = {(X₁ -X₂)² + (Y₁ -Y₂)²}^1/2 It represents. Also, the hole H₁ Center of (X₁ , Y₁) And hole H₃ 
Center of (X₃ , Y₃) Between l₃ And l₃ = {(X₂ -X₃)² + (Y₂ -Y₃)²}^1/2 Express with. S shown in FIG.₁ , S₂ , S₃ , Β
₃ , L₂ , L₃ In addition, the area S of the entire component is
It is used as a reference quantity that represents the characteristics. In other words, certain parts are
Whether or not it is the target part B is determined from the image of the part.
s₁', S₂', S₃', Β₃', L₂', L₃Calculate ', S'
And these are the reference parameters s₁ , S₂ , S₃ , Β ₃ ，
l₂ , L₃ , S to judge. these
The reference parameters of are calculated in advance and stored in the RAM 52.
Keep it. The image processing unit 6 uses the reference parameter.
4 and the reference parameter in the RAM 52 is the control device.
When the power of 17 is turned on, the image processing unit 64
It is transferred to the port RAM 61.Part recognition Fig. 24 shows images captured by the camera 2 and formed into parts.
Diagram explaining the principle of calculating the center of gravity (center) of the created hole
Is. As mentioned above, the parts are stored in cells in the pallet.
Has been paid. The area setting unit 74 in FIG.
Cut the image of the whole cell of the stored palette.
Has a function to put out.

In FIG. 24A, the image of the part is captured by the camera 2 and the image is set to "1" depending on the brightness level.
5 is a diagram schematically showing that the result of binarizing "0" and "0" is stored in the frame memory 75. FIG. Each cell in (a) corresponds to one CCD element and also corresponds to one memory in the frame memory 75. It is delicate whether the edge portion of the hole is quantized as "1" or "0", but in this embodiment, the maximum brightness is 50.
A cell of% or more is "1", and a cell of less than 50% is "0".

Next, the labeling processing method for judging that the image shown in FIG. 24 (a) is a hole will be described. From the beginning of the cell, determine 0/1 of the concentration for each cell,
When a "1" is found, it is determined whether or not there is a block in which a "1" is found in the block portion composed of 3 × 3 cells before the "1" is found. If the found "1" cell is the first "1" cell, the "1" cell is labeled. Similar processing is performed for all cells to be processed. This way,
A plurality of closed regions consisting of "1" cells are created. Then, each closed region is labeled. Taking the part B in FIG. 22 as an example, the holes "H" are respectively provided in the three holes.
It becomes possible to label as ₁ ”, hole“ H ₂ ”, and hole“ H ₃ ”.

Next, how to obtain the areas s ₁ ′ to s ₃ ′ and the center position (center of gravity position) of the labeled holes will be described with reference to FIG. First, for the area of the hole, the number of "1" cells in the labeled and bound area is counted, and the total is taken as the area. Next, when the center of the hole is to be obtained, a histogram of the number of cells which is "1" is created for each of the X and Y directions of the image of FIG. The position is the center of gravity (X ₁ ', Y ₁ ') of the hole H ₁ .

If the above processing is performed on the images of the three holes, the areas s ₁ ', s ₂ ', s ₃ 'of the three holes of the part to be gripped and the CCD coordinates of the three holes are obtained. Coordinates of the center of the system (X ₁ ', Y ₁ '), (X ₂ ', Y ₂ '), (X ₃ ',
Y ₃ ') can be obtained. This calculated value and the second
As shown in FIG. 3, by comparing with the reference value of the characteristic amount stored in advance, it is possible to judge whether or not the component is correct.

In other words, as a comparison of feature quantities, s₁ ≒ s₁',
s₂ ≒ s₂', S₃ ≒ s₃', L₂ ≒ l ₂', L₃ ≒ l₃', Β
₃ = Β₃'To do.Coordinate transformation Next, regarding the conversion of the coordinate system of image processing and the coordinates of the robot,
Explain.

FIG. 25 shows X _R Y _R of the robot coordinate system.
And the visual coordinate system X _V Y _V. The target position in the robot coordinate system is (P _xR , P _YR , 1), and the target position in the visual coordinate system is (P _xV , P _YV , 1). Furthermore, the coordinate system of the robot X _R Y origin of _R O _R
And the deviation of the origin O _V of the visual coordinate system X _V Y _V is (δ _x ,
δ _Y ), assuming that the robot coordinate system X _R Y _r is rotated by θ degrees and the resolution of the visual system is kmm / pixel,

[0120]

[Equation 1] Have a relationship.

The conversion parameters are RAM 52 shown in FIG.
Stored in advance in the two-port RAM 61 when the control device 17 is turned on, and the image processing unit 64 uses this parameter to convert the position information of the component when coordinate conversion is performed. Is converted from the image coordinate system to the robot coordinate system or from the robot coordinate system to the image coordinate system.

FIG. 26 shows a preparatory operation routine until the controller 17 is powered on and the keyboard 20 is activated (the activation program 10 of FIG. 5).
It is a flowchart chart of 0).

First, the power supply to the control unit 17 is step S.
If entered in 400 steps, step S401
Then, the reference value of the characteristic parameter is set to the instruction interpreter 8 for each part.
From inside RAM 52 of 7 through buses 86 and 59,
The data is transferred to a predetermined area of the two-port RAM 61. At this time, it goes without saying that the data in the RAM 52 remains unchanged.

Next, in step S402, the above-mentioned conversion parameters used for the coordinate conversion calculation are also transferred from the RAM 52 to the two-port RAM 61.

Next, in step S403 of the step, X
The origin of the Y table 3 is set. The table 3 is for moving the visual camera 2 and the lens in a plane. Next, in steps S404 and S405,
The origins of the robot 6 and the shuttle 9 are set, and in step S406, the apparatus is ready to start.

Needless to say, after the power is turned on, the keyboard input is not accepted until the process goes to step S406. <Effects of Embodiment> According to the embodiment described above, the following effects can be obtained. That is, in the control device 17, one instruction interpreting section 87 interprets two different types of instructions, that is, a robot operation instruction for assembling and an image processing instruction for performing image processing. On the other hand, the actual execution of the robot operation is controlled by the assembly operation control (that is, the robot operation control unit 62, the hand operation control unit 63), and the image processing is performed by the image processing unit 64. I have to.
In other words, two different processes, robot motion and image processing, are described by two modularized programs,
One instruction interpreting unit performs those program modules, while the robot motion and the image processing are performed by the assembly motion control unit (that is, the robot motion control unit 6).
2, the hand operation control unit 63) performs the image processing, and the image processing unit 64 performs the image processing.

Therefore, there is an effect that the development load of the control device 17 can be greatly reduced. Because the instruction interpretation is unified, the arbitration control and the synchronization control of the two program modules that operate in parallel can be easily performed. Also, since the instruction interpreting unit has the same specifications, there is an effect that the device can be easily used by the user even after the factory is installed. Furthermore, since the command interpreting means are the same, the communication interface and communication program between the respective devices are also simplified as the respective commands, and the devices are inexpensive and can be used more easily for the user. .. Further, since the execution of the robot operation and the execution of the image processing are performed in two lines, the development of the control procedure in the assembling operation control unit can be performed independently from the development of the control procedure in the image processing unit 64, which is efficient. Because there is.

Specifically, in the control device 17, one instruction interpreting section 87 is composed of an input / output means including a CPU 50, memories (51, 52), a keyboard 20, and a CRT 19. Then, they are connected by a local bus 86 capable of communicating only within the command interpreting section 87, and the CPU 50, the robot operation control section 62, the hand operation control section 63, and the image processing section 64 are separate from the local bus 86. Are connected by a common bus 59. Moreover, the common bus 59 is a multi-bus according to a general standard. Further, in the ROM 51 which is a part of the memory of the instruction interpreting unit 87, an instruction interpreting program and a multitasking OS or a program for appropriately using the instruction interpreting program by a robot, a hand operation instruction, and an image processing instruction are stored. Programs that support output are stored in advance.
The RAM 52, which is a random access memory of the instruction interpreting unit 87, is divided into areas and these programs are stored in advance. Further, the robot and hand operation commands and the image processing commands can be separately input by an input means such as a keyboard. Then, when the assembly apparatus is activated by an input means such as a keyboard, the control apparatus first interprets and executes the operation command of the robot and the hand, and starts image processing during the operation command of the robot. When the CPU for the instruction interpreting unit finds the instruction to be executed, the CPU of the instruction interpreting unit sequentially starts interpreting and executing the image processing instruction, and finds the instruction to end the processing in the image processing instruction, and thus the CPU executes the image instruction processing. The operation is stopped and only the robot command is interpreted.

Therefore, for example, an operation such as capturing an image at a timing when the robot arm escapes from the camera area can be easily described. Further, since the timing can be performed via the shared memory 2-PORT-RAM, it is possible to arbitrate the access time level of the computer at a very high speed. When the image processing is not necessary, the CPU of the instruction interpreting unit can concentrate on only the instruction interpretation of the robot operation, and a single CPU performs a plurality of processes. However, this does not cause a delay in processing time. It was

Further, since the instruction interpreting means has the same specifications, there is an effect that the apparatus can be easily used by the user even after the factory is installed.

Further, since the command interpreting means are the same, the communication interface and the communication program between the respective devices are also simplified as respective commands, and the device is inexpensive and can be used more easily for the user. There is. : Instruction codes for the operation control units 62 and 63, instructions for the image processing unit 64, and the like have a unified format instruction code (see FIGS. 13 to 15)
Figure) and stored in the memory 60 or the RAM 61. That is, a medium called an instruction code and a memory exists between the upper control unit (interpretation unit 87) and the lower control unit (operation control units 62 and 63), and the upper control unit (interpretation unit 8).
It absorbs the difference in instructions between the robot task and the vision task in 7). : Since a command (START) for activating a vision task which is a child task is prepared in the robot task which is the main task, arbitration of activation between both tasks becomes easy.
Furthermore, since a timing confirmation command and the like are prepared, it becomes easy to start and stop between tasks. : Various parameters such as coordinate conversion parameters and reference feature parameters are stored in the battery backed up RAM.
Since these parameters are stored in the RAM 61 and are transferred to the required RAM 61 when the power is turned on, unified management of these parameters becomes easy. Specifically: -1: All can be done by providing only one input / output means such as a keyboard and a CRT floppy disk drive, and at a portion required to input / output operation commands such as a robot.

-2: The RAM of the image processing unit need not be backed up.

-3: It is not necessary to input the parameters again even if the image processing unit is replaced due to a failure or the like.

-4: Since it is not necessary to store parameters twice, there is no malfunction due to misalignment or the like. <Modifications> The present invention can be variously modified without departing from the spirit thereof. For example, in the above embodiment, as shown in FIG. 4, the workpieces arranged in the pallet in the shape of a matrix have been described. They may be arranged side by side or randomly stored in a pallet. In short, it suffices that the position of the work in the pallet is roughly determined, and it is important to be able to teach the robot the approximate position of each work. In addition, in the above-mentioned embodiment, the size of the cell is constant, and the work is one kind. The present invention can be applied even when the sizes of cells are different and different works are present in one pallet. In this case, the controller 17 may send the correspondence between the position of the cell, the size of the cell, and the type of work in the cell to the image processing unit 64. The present invention can also be applied to the case where a plurality of works are present in one cell. : In the above embodiment, the coordinate conversion is performed by the visual system, but it may be performed by the robot controller side. In addition, in the above embodiment, the center of gravity of the holes A and B is the second position.
As described with reference to FIG. 4, the position where the maximum value of the histogram is taken is defined as the barycentric position, but it may be set as follows. That is, in FIG. 24, when the number of all dots is n, the point at which the cumulative frequency is n / 2 is the center of gravity. According to this technique,
The center of gravity can be calculated for holes and the like that are not of interest. In the above-described embodiment, the characteristic parameter and the coordinate conversion parameter are always stored in the memory 52 of the instruction interpreting unit 87, and the 2-PO accessible by the image processing unit after the power is turned on.
The data is transferred to the RT-RAM. The present invention is not limited to this, and for example, the parameters uniquely used by the image, such as the absolute start position of image capture and the gain of the input of the camera 2, are stored in the memory of the command interpreting unit, and the same as above. May be forwarded to. Further, the parameters required for the robot operation unit and the hand operation unit may be similarly transferred from the memory of the instruction interpretation unit to the shared memory. In the above embodiment, the operation of the robot is first activated, and the image processing is activated within the operation command of the robot. However, the processing time of the image is long, and the apparatus for intermittently moving the assembling apparatus. In the case of application to etc., there is no problem even if it is configured so that the operation instruction interpretation of the robot is activated from the image processing instruction and the stop instruction is inserted in the robot operation instruction so that the CPU processing is not performed. . : In the above embodiment, if there is a sequence control command (jump command, repeat command, timer command) other than the command code to the robot motion unit and the image processing unit in the series of motion and processing commands, command interpretation What is understood by the means and executed by the instruction interpreting unit itself is line number 3 in FIG.
It can be easily considered from the description of the 60 vision task start commands. In the above-mentioned embodiment, the image processing unit is a means for transmitting and receiving the instruction code from the instruction interpreting unit to the 2PORT-RAM.
Although it is set to 61, it is also possible to use the shared memory 60 like the robot operation unit of the above-mentioned embodiment.

Similarly, in the above-described embodiment, the shared memory 60 is used as the means for the robot operation unit and the hand operation unit to send and receive the instruction code from the instruction interpretation unit, but like the image processing unit of the above-described embodiment, 2PORT- is used. It is also possible to use RAM. It is stored in the unit and is transferred from this storage unit to the gripping unit and / or the image processing unit in which it is used, if necessary. For this reason, there is only one place for storing the parameters, and therefore the management thereof can be unified.

[0136]

As described above, according to the control device of the visual grasping automatic gripping device of the present invention, the parameters are
The program including the image processing instruction and the gripping operation instruction is stored in the same storage unit, and is transferred from the storage unit to the gripping unit and / or the image processing unit in which the program is used, as necessary. For this reason, there is only one place for storing the parameters, and therefore the management thereof can be unified.

According to a preferred aspect of the present invention, the parameter is a characteristic parameter for recognizing the shape of the workpiece to be gripped.

According to a preferred aspect of the present invention, the parameter is a coordinate conversion parameter between the visual coordinate system of the image processing section and the coordinate system of the grip section.

According to a preferred aspect of the present invention, the transfer means transfers the characteristic parameter to the memory of the image processing unit after the control device is turned on and before the gripping operation is started. To do.

According to a preferred aspect of the present invention, the memory is a two-port RAM.

Thus, according to the present invention, specifically, for example, by providing only one input / output means such as a keyboard and a CRT floppy disk drive, and at a portion where input / output of operation commands such as a robot is required. , You can do everything. The RAM of the image processing unit need not be backed up. Even if the image processing unit is replaced due to a failure, it is not necessary to input the parameters again. Since it is not necessary to store parameters twice, there is no malfunction due to misalignment.

[Brief description of drawings]

FIG. 1 is a perspective view of a robot assembly system according to a preferred embodiment to which the present invention is applied.

FIG. 2 is a block diagram showing the configuration of a control device 17 of the system shown in FIG.

FIG. 3 is a block diagram of an image processing unit 64 of the control device 17.

FIG. 4 is a plan view showing the structure of a pallet used in the system shown in FIG.

FIG. 5 is a diagram showing a data configuration of a ROM 51 of an interpretation unit 87.

FIG. 6 is a diagram showing a data configuration of a ROM 52 of an interpretation unit 87.

[Fig. 7]

FIG. 8 is a diagram specifically showing the operations of the shuttle and the robot of the system shown in FIG. 1;

[FIG. 9]

FIG. 10 is a flow chart of a program describing a robot task as an example.

FIG. 11 is a flow chart of a program describing a vision task as an example.

FIG. 12 is a diagram schematically showing how a robot program is converted into an instruction code by an interpretation unit 87 and the instruction code is executed.

FIG. 13

FIG. 14

FIG. 15 is a diagram showing an instruction code format.

FIG. 16 is a diagram showing a relationship between an interpreting program and each task, and also showing flags necessary for controlling the interpretation therein.

17A],

FIG. 17B is a flowchart showing an interpretation procedure of the interpretation unit 87.

FIG. 18 is a flow chart showing a control procedure in the robot / hand operation control unit.

FIG. 19 is a flow chart showing a control procedure of the image processing unit 64.

FIG. 20:

FIG. 21 is a timing chart showing the movement of each part when the program of FIGS. 9 to 11 is executed.

FIG. 22:

FIG. 23 is a diagram illustrating the definition of characteristic parameters in component recognition.

FIG. 24 is a view for explaining the principle of detection of the position of the center of gravity necessary for component recognition.

FIG. 25 is a diagram showing a relationship between a visual coordinate system and a robot coordinate system.

FIG. 26 is a flowchart showing the control procedure of the startup program 100.

[Explanation of symbols]

 2 CCD camera 3 XY table 6 Robot 7 Hand 8 Assembly stage 9 Shuttle 11, 12 Parts supply section 13a, 13b Pallet 17 Control section 18 Cable 19 CRT 50 CPU 51 ROM 52 RAM 60 Shared memory 61 Two-port RAM 62 Robot operation Control unit 63 Hand operation control unit 64 Image processing unit 87 Command interpretation unit

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification number Office reference number FI technical display location G05B 19/18 H 9064-3H 19/19 H 9064-3H 19/403 S 9064-3H (72) Inventor Yu Shibata 3-30-2 Shimomaruko, Ota-ku, Tokyo Canon Inc.

Claims (5)

[Claims] 1. An automatic gripping device comprising an image processing unit for picking up an image of a work and calculating position information of the work from this image, and a gripping unit for gripping the work based on the position information. A control device, a program including a series of image processing commands for imaging the work and calculating position information from an image of the work, and a series of gripping operation commands for gripping the work based on the position information; A storage unit for storing a parameter for calculating the position information of, and interpreting the image processing command and the gripping operation command in this program while distinguishing and interpreting the image processing command to the image processing unit, One interpreting unit for passing the gripping operation command to the gripping unit, and the parameter, prior to the execution of the program,
A control device for an automatic gripping device using vision, comprising: a transfer unit that transfers the image to the image processing part or the gripping part. 2. The control device for an automatic visual grasping device according to claim 1, wherein the parameter is a characteristic parameter for recognizing the shape of a workpiece to be grasped. 3. The visual sense according to claim 1, wherein the parameter is a coordinate conversion parameter between the visual coordinate system of the image processing unit and the coordinate system of the gripping unit. The control device for the automatic gripping device. 4. The transfer means sets the characteristic parameter to
The automatic visual grasping device according to claim 1, wherein after the control device is turned on, it is transferred to the memory of the image processing unit before the grasping operation is activated. Control device. 5. The control device for an automatic visual grasping device according to claim 4, wherein the memory is a two-port RAM.