JPH0523981A - Robot assembler - Google Patents

Abstract

PURPOSE:To compensate and cope with an error at once after it has happened. CONSTITUTION:A recognitive part 3 is operated even during assembly operation. This recognitive part 3 recognizes every condition of a robot 1 and parts through an image, and a selecting part 5 or the like collates it with an intermediate state. In the case where some error happening is recognized as a result of the collation, the robot 1 is controlled so as to compensate this error. Thus, the error happening is recognized, the selecting part 5 stops the assembly operation at once.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot assembling apparatus using a recognition unit.

[0002]

2. Description of the Related Art A robot assembling apparatus is expected to promote automation and efficiency of a production line and to perform more intelligent processing. For example, (1) Ohtake et al., “Prototype of motion planning system using work model”, Information Processing Society of Japan “Knowledge Engineering and Artificial Intelligence” study group, 53-5 (1987) ), (2) Ohtake et al., “Planning System for Assembly Intelligent Robot ARI”, Journal of Information Processing Society of Japan, Vol. 31, No. 3 (1990) and the like.

These devices estimate the structure of the structure presented as a sample by stereovision, estimate the assembling method, and assemble the same structure. That is, it has a function of recognizing the shape of parts, a function of creating an assembly plan, and the like. More specifically, the recognition unit obtains the position and orientation of each part before the start of work and after the end of work (completion of assembly) before performing the assembly work, and automatically inputs the initial state and the target state to the assembly device. Play a role in.

[0004]

However, in such a conventional apparatus, when an error or an error in the position or orientation of the part or the robot occurs during the assembly work, the error is detected and a corresponding process is performed. I couldn't. This is used only before the recognition unit that recognizes the position and orientation of the part or robot, before performing the assembly work,
It is not used during the assembly work.

The present invention has been made to solve the above problems, and when an error in the position and orientation of a component or a robot occurs during the assembly work, it can be corrected and therefore a more accurate assembly can be achieved. Is possible,
It is an object of the present invention to provide an assembling device using a robot, which can stop work or the like when an error occurs.

[0006]

In order to achieve such an object, the present invention provides a robot capable of moving a plurality of parts to be assembled, a robot and / or a position and / or posture of the parts. Means for recognizing an image when the robot is stationary, means for planning a motion sequence of the robot, at least an initial state of the arrangement of the parts, a target state after the parts are assembled, and a support relationship between the parts in the initial state. (Initial state support relation), support relation between the parts in the target state (target state support relation), intermediate state indicating the desired arrangement of the parts at a predetermined point during assembly work, robot and model that is the shape of the part Means for storing information related to the intermediate state, and information related to the intermediate state is sequentially updated based on the stored information. While collating the result of the recognition, in accordance with the plans to be a predetermined state position of the robot and /
Alternatively, a means for correcting / controlling the posture is provided, and the assembly work is executed while monitoring the state of the parts and the operation result of the robot.

[0007]

In the present invention, the assembly work is executed by moving the plurality of parts to be assembled by the robot in accordance with the planned operation sequence. Furthermore, in the present invention, the image recognition is not only initially performed, but also sequentially performed in the assembly work. That is, the position and / or position of the robot and / or the component at a predetermined stationary time during the assembly work, such as when the robot grips the component.
Alternatively, the posture is image-recognized. The result is the actual position and orientation of the robot and parts. For example, it is information such as the presence / absence of a component that should have been gripped.

On the other hand, in the present invention, the information regarding the intermediate state is sequentially updated based on at least one of the initial state, the target state, the initial state support relationship, the target state support relationship, the intermediate state, and the model. The intermediate state thus obtained shows the desired arrangement of each component.

According to the present invention, the intermediate state and the result of image recognition are collated, and the position and / or orientation of the robot is corrected and controlled according to a plan so as to obtain a predetermined state. That is, the assembly work is executed while monitoring the state of the parts and the operation result of the robot. As a result, if an error such as a displacement of the robot occurs during work, this will be detected by image recognition and the necessary correction operation will be executed, and an error such as removal of a part will occur during work. Even in case of doing so, it is possible to take a corresponding action such as stopping work or re-planning.

[0010]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

(1) Construction of Embodiment FIG. 1 shows the construction of a robot assembly apparatus according to an embodiment of the present invention. It is the robot 1 that performs the assembly work with the apparatus shown in this figure. The operation of the robot 1 is controlled by the control unit 2. The recognition unit 3 has a function of recognizing a component or a hand of the robot 1, and includes recognition units 3-1, 3-2 and 3-3 as described later. It is the camera 4 that captures the image information of the part or the hand of the robot 1 and supplies it to the recognition unit 3. Further, the selection unit 5 performs a necessary operation according to the supplied signal, the support relationship calculation unit 6 calculates the support relationship between the parts, and the planning unit 7 plans the operation sequence of the robot 1. The work storage unit 8 is a means for storing the initial state and the like, and the model storage unit 9 is a means for storing the model of each component. The input unit 10 is means for a person to input these pieces of information into the work storage unit 8 and the model storage unit 9 from the outside.

(2) Functions of each part of the embodiment The device of this embodiment realizes a high positioning accuracy by the robot 1, executes a countermeasure against an error or an error, and further makes the assembly work autonomous and human. There is an advantage in realizing labor saving. That is, when the robot 1 performs the assembling work, in this embodiment, the assembling work is performed while confirming whether the assembling work is correctly performed. If an error or an error occurs during the assembling work, the assembling apparatus performs the assembling work. Stop and correct the error. In addition, the apparatus of this embodiment has the ability to automatically input the initial state, the target state, the initial state support relationship, and the target state support relationship in order to autonomously perform the assembly work and reduce the labor of the human. There is.

Here, the initial state means the position and posture of each component before the start of the assembling work. On the other hand, the target state means the position and posture of each component after the assembly work is completed (assembly is completed). In addition, the support relationship refers to the relationship between each part and the part that supports the part, and is calculated by estimating the relationship from the surfaces in contact between the parts and the normal direction of the surface. Is. What is the initial state support relationship?
The support relationship between the parts in the initial state means the support relationship between the parts in the target state.

All of this information is stored in the work storage unit 8
Information stored in. When the assembly work is performed by the apparatus of this embodiment, the work storage unit 8 and the model storage unit 9
It is necessary to store in advance the information necessary for. There are six types of information stored in the work storage unit 8: A) initial state B) target state C) initial state support relationship D) target state support relationship E) intermediate state F) work environment knowledge.

Of these, A) and B) are information that can be input from the input unit 10 and can also be input from the recognition unit 3-1. That is, the initial state and the target state are information that can be selectively input from both the recognition unit 3-1 and the input unit 10. Further, C) and D) are the supporting relationship calculation unit 6
Alternatively, it is information input from the input unit 10.

Further, the intermediate state means the position and posture of the robot 1 during assembly work and the position and posture of each part during assembly work. this is,
This is information relating to the characteristics of this embodiment. This information is sequentially updated and collated with the result of image recognition. As a result of the collation, an error or an error is detected, and a corresponding correction operation or the like is executed. In addition, the work environment knowledge is information such as the shape of the robot 1, the operation range of the robot 1, the work table, and obstacles that are not targets of the work, but are necessary for the planning unit 7 to plan the operation sequence of the robot 1. Say. This is input from the input unit 10.

Then, the model storage unit 9 represents the shape of each part as a model of the object to be recognized and information about the shape of the hand of the robot 1 by a wire model used in the field of CAD / CAM, for example. Remembered in. This is input from the input unit 10.

On the other hand, in this embodiment, the recognition unit 3 is provided with three recognition units 3-1 to 3-3. Of these, the recognition unit 3
-1 obtains the position and orientation of the recognition target object by using the model of the recognition target object (component) stored in the model storage unit 9. This is used for inputting the initial state and the target state and storing them in the work storage unit 8.

The recognition unit 3-2 includes the recognition target object and the model of the hand of the robot 1 stored in the model storage unit 9 and the recognition target object and the robot stored in the working storage unit 8 as an intermediate state. Using the predicted position and orientation of the first hand, it is confirmed that the recognition target and the hand of the robot 1 actually exist within a certain range including this position and orientation. This is used to confirm the operation of the robot 1.

The recognition unit 3-3 is then connected to the model storage unit 9
Using the model of the recognition target object and the hand of the robot 1 stored in, and the predicted position and orientation of the recognition target object and the hand of the robot 1 stored in the work storage unit 8 as an intermediate state, After confirming that the recognition object and the hand of the robot 1 actually exist within a certain range including the position and the attitude, the actual position and attitude of the recognition object and the hand of the robot 1 are obtained. This is used for checking the assembly of the robot 1, correcting the error of the robot 1, and correcting the error in the intermediate state.

When the recognition units 3-2 and 3-3 cannot confirm the recognition target object and the hand of the robot 1, in this embodiment, it is determined that an error has occurred during the assembly work. Stop working. Further, when there is an error between the actual position and orientation of the recognition target object and the hand of the robot 1 obtained by the recognition unit 3-3 and the predicted position and orientation of the recognition target object and the hand of the robot 1. , Judge that an error has occurred during the assembly work and correct the error.

(3) Operation of the Embodiment Next, the operation of this embodiment having such a configuration and function will be described.

(3.1) From Input to Output of Operation Signal FIG. 2 shows a flow of operation from input to output of operation signal in this embodiment. In this embodiment, first, the user needs to input necessary information from the input unit 10 and store it in the work storage unit 8 and the model storage unit 9 (101). The information to be input includes an initial state, a target state, an initial state support relationship, a target state support relationship, work environment knowledge, and a model of each part. As described above, the last one of these is stored in the work storage unit 8,
The model of each component is stored in the model storage unit 9. However, regarding the initial state, the target state, the initial state support relationship, and the target state support relationship, the recognition unit 3-1 or the support relationship calculation unit 6
Is available and does not necessarily have to be input from the input unit 10. When the input is completed, the input unit 10 sends a signal to the planning unit 7 (102).

When a signal is sent from the input unit 10 to the planning unit 7, the planning unit 7 stores the initial state, the target state, the initial state support relation, the target state support relation, and the work environment knowledge in the work storage unit 8.
The model storage unit 9 is inquired about the model of each component (10
3). Here, when the initial state and the target state are not stored in the work storage unit 8, the work storage unit 8 activates the recognition unit 3-1 (104). The recognition unit 3-1 is the model storage unit 9
Using the model of each part stored in
The image information from the image is recognized (105), and the initial state and the target state obtained as a result of the recognition are input to the work storage unit 8 (1
06).

When the initial state support relation and the target state support relation are not stored in the work storage unit 8, the support relation calculation unit 6 causes the initial state and the target state on the work storage unit 8 and the model storage. The initial state support relationship and the target state support relationship are calculated based on the model of each part on the part 9 (1
07), the initial state support relationship and the target state support relationship are input to the work storage unit 8 (108).

After that, the planning unit 7 calculates an assembly procedure, a grasped position of each part, etc. from the initial state, the target state, the initial state support relationship, the target state support relationship, the work environment knowledge, and the model of each part, Plan the motion sequence of the robot 1 (10
9).

The robot 1 carries out an assembling work by sequentially executing the operations of the operation train planned by the planning unit 7 one by one. This work is performed based on the operation signal supplied from the planning unit 7 to the selection unit 5. That is, after planning the operation sequence, the planning unit 7 outputs an operation signal to the selection unit 5 (11
0), the selection unit 5 determines the operation signal sent from the planning unit 7 and executes the corresponding operation. Here, the motion signal is a signal including information such as the motion of the robot 1, the movement amount of the robot 1, the operation target component of the robot 1, and the like.

(3.2) Operation According to Operation Signal The operation of the embodiment apparatus when the robot 1 executes one operation will be described below. Here, an initial state in which two parts and are arranged as shown in FIG. 3A is assumed, and based on an example of assembling this into a target state as shown in FIG. I will explain.

The assembling work of the parts is roughly classified into a pick work and a place work. Of these, the picking operation is an operation of gripping a component by the robot 1, and the placing operation is an operation of placing a component.

From the initial state shown in FIG.
A series of assembling procedures to reach the target state shown in (b) are as follows. First, a picking operation as shown in FIG. 4 is performed.

In this work, the robot 1 first grabs a part. This is an operation from FIG. 4A to FIG. 4B, and this is called an approach operation. At this time, in the present embodiment, the error in the intermediate state is corrected for the component by the method described later. This is because the robot 1 correctly grasps the component even when the component has a position and orientation different from the intermediate state position and orientation (in this case, the same as the initial state position and orientation) due to unexpected movement. This is so that

Next, a grasp operation is performed. This is an operation in which the robot 1 approaching a part grabs the part, and is an operation from FIG. 4 (b) to FIG. 4 (c). In this case, the assembling apparatus corrects the error of the robot 1. This is to realize high positioning accuracy of the robot 1.

Further, the robot 1 lifts the component. This is called department store operation. 4 (c) to 4
This is the operation up to (d). In this case, the apparatus of this embodiment confirms the operation of the robot 1 with respect to the parts. this is,
This is to promptly detect an error and stop the work when the robot 1 cannot correctly grasp the component.

Next, the work of placing the parts is executed. This place work is composed of three operations: approach, release and department store. The approach operation of the place work is the operation from FIG. 4 (d) to FIG. 5 (a). In this case, unlike the approach operation when placing a component (described later), since there is no contacting component with respect to the component, error correction in the intermediate state is not performed.

The operation from FIG. 5 (a) to FIG. 5 (b) is as follows.
It is a release operation. That is, the robot 1 releases the parts. Then, the department store operation from FIG. 5B to FIG. 5C, that is, the operation of lifting the robot 1 without any parts is executed. In the case of this figure, unlike the department store operation when placing a part to be described later, since there is no contact part with the part, the operation check of the robot 1 is performed not for the assembly check but for the part.
This is to promptly detect an error and stop the work when the robot 1 cannot place the parts correctly.

Further, the picking work of the parts is executed. That is, the approach operation from FIG. 5 (c) to FIG. 6 (a), the grasp operation from FIG. 6 (a) to FIG. 6 (b), and the department store operation from FIG. 6 (b) to FIG. 6 (c). Executed. In the approach operation, the error in the intermediate state is corrected for the part. This is because even if the part is moving for some reason and the position and attitude are different from the position and attitude in the intermediate state (in this case, the position and attitude are the same as the initial state), the robot 1 correctly detects the part. This is so that it can be grabbed. In the grasp operation, the error of the robot 1 is corrected. This is to realize high positioning accuracy of the robot 1. In the department store operation, the operation of the robot 1 is checked for parts. This is to promptly detect an error and stop the work when the robot 1 cannot correctly grasp the part.

Then, a work of placing parts is performed. This work also consists of approach, release, and department store operations. 6 (c) to 7
In the approach operation up to (a), since there is a contacting part (part) with respect to the part, the error in the intermediate state is corrected for the part. This is because even if the part is moving for some reason and has a position and orientation different from the position and orientation in the intermediate state (in this case, the position and orientation are the same as the target state), the robot 1 can This is so that the can be assembled correctly. Figure 7
Following the release operation from FIG. 7A to FIG.
The department store operation from (b) to FIG. 7 (c) is executed. In this department store operation, since there is a contact component (component) with respect to the component, the assembly confirmation of the robot 1 is performed for each component. This is to promptly detect an error and stop the work when the robot 1 cannot assemble the parts correctly.

As described above, in each operation of the assembly work performed by using the apparatus of this embodiment, the operation check of the robot 1, the assembly check of the robot 1, the error correction of the robot 1, the intermediate state Characteristic operations such as error correction are executed. In these operations, the recognizing unit 3 operates, and a stationary robot or a part at the boundary of a series of operations is image-recognized. Next, details of the basic operation of the present embodiment will be described, and the characteristic operations thereof will be described.

(3.2A) When the operation signal indicates the approach operation of the pick work In this case, the apparatus of the embodiment executes the operation of the flow as shown in FIG.

First, the selection unit 5 outputs the operation signal to the work storage unit 8
To (201). The work storage unit 8 updates the intermediate state based on the operation signal sent from the selection unit 5 (20
2). In this step 202, more specifically,
The work storage unit 8 calculates the position and orientation of the robot 1 based on the operation signal, calculates the position and orientation of each component from the operation of the robot 1, and the position of each component in the intermediate state stored by the work storage unit 8. And update the posture.

When the update of the intermediate state is completed, the work storage unit 8
Activates the recognition unit 3-3 to correct the error in the intermediate state. The object of this correction is a component that the robot 1 operates. For example, the approach operation shown in FIGS. 4A and 4B is a part, and the approach operation shown in FIGS. 5C to 6A is a part. This behavior is
More specifically, it is performed as follows.

First, the recognition unit 3-3 uses the model of the correction target object stored in the model storage unit 9 and the predicted position and orientation of the correction target object stored in the working storage unit 8 as an intermediate state. And are used to recognize the image information from the camera 4 and obtain the position and orientation of the correction target (20
3). At this time, when the recognition unit 3-3 cannot confirm the correction target, the recognition unit 3-3 determines that an error has occurred during the assembly work and sends a signal to the selection unit 5 (204 ), The selection unit 5 stops the assembly work (205). When the recognition unit 3-3 can confirm the correction target object, the recognition unit 3-3 sends the obtained position and orientation of the correction target object to the work storage unit 8 (206). The work storage unit 8 obtains an error between the position and orientation of the correction target sent from the recognition unit 3-3 and the predicted position and orientation of the correction target stored in the intermediate state (20).
7). If there is an error, the work storage unit 8 corrects the position and orientation of the correction target object stored as the intermediate state, and accordingly corrects the position and orientation of the robot 1 stored as the intermediate state ( 208).

In this way, when the error correction of the intermediate state is completed, the work storage unit 8 sends the position and orientation of the robot 1 stored as the intermediate state to the control unit 2 (209).
The control unit 2 moves the robot 1 to the position and posture of the robot 1 sent from the work storage unit 8 (210). When the operation of the robot 1 is completed, the control unit 2 sends a signal to the selection unit 5 (211), and the selection unit 5 sends a signal to the planning unit 7 (21).
2). In this way, the approaching motion of the picking work is completed.

As described above, in the approach operation of the pick work in this embodiment, the intermediate state is corrected according to the image recognition result by the recognition unit 3-3. As a result, the robot 1 can approach the parts to be picked more accurately. If the recognition unit 3-3 cannot recognize the component, the assembly work by the robot 1 is stopped. This enables quick and accurate work recovery. Further, instead of the work stoppage related to step 205, a signal may be issued to the planning unit 7 to execute the re-planning, and in this case, the autonomous apparatus has better usability.

(3.2B) When the motion signal indicates the grasp motion of the picking work This motion is the motion shown in FIGS. 4 (b) to 4 (c) and 6 (a) to 6 (b). That is, this is an operation for grasping the component executed after the approach operation. In this operation, the selection unit 5 first activates the recognition unit 3-3 to correct the error of the robot 1.

This error correction operation is performed as shown in FIG. First, the recognition unit 3-3, the model of the hand of the robot 1 stored in the model storage unit 9, the predicted position and orientation of the hand of the robot 1 stored in the intermediate state of the work storage unit 8, Is used to recognize the image information from the camera 4 and obtain the position and orientation of the hand of the robot 1 (301). However, when the recognition unit 3-3 cannot confirm the hand of the robot 1, the recognition unit 3-3
-3 determines that an error has occurred during the assembly work and sends a signal to the selection unit 5 (302), and the selection unit 5 stops the assembly work (303). The recognition unit 3-3 is the robot 1
If the hand of the player can be confirmed, the recognition unit 3-3
Sends the position and orientation of the hand of the robot 1 to the selection unit 5 (304). The selection unit 5 sets the predicted position and posture of the hand of the robot 1 stored in the intermediate state,
The error of the position and orientation of the hand of the robot 1 sent from the recognition unit 3-3 is calculated (305). If there is an error, the selection unit 5 sends the error to the control unit 2 (30
6). The control unit 2 corrects the position and orientation of the robot 1 by the error sent from the selection unit 5 (307).

When the error correction of the robot 1 is completed in this way, the controller 2 closes the hand of the robot 1 (308). When the operation of the robot 1 ends, the control unit 2 sends a signal to the selection unit 5 (309). The selection unit 5 is the planning unit 7
(310), and the grasping operation of the picking work is completed.

As described above, in the grasp operation of the picking work in this embodiment, the error of the robot 1 is corrected according to the image recognition result by the recognition unit 3-3. As a result, the grasp operation can be accurately performed even when the robot 1 does not accurately approach the parts. Further, when the recognition unit 3-3 cannot recognize the hand of the robot 1, the assembly work by the robot 1 is stopped as in the approach operation. The advantages of this outage mentioned in the description of the approach operation can be enjoyed as well and re-planning is possible as well.

(3.2C) When the operation signal indicates the department store operation of the pick work The department store operation of the pick work is shown in FIGS. 4 (c) to 4 (d).
Alternatively, the operation is as shown in FIGS. 6 (b) to 6 (c). This is an operation to lift a component while grasping it. In the present embodiment, the operation confirmation of the robot 1 is executed in the department store operation of the picking work.

As shown in FIG. 10, in the department store operation, first, the selection unit 5 sends an operation signal to the work storage unit 8 (401). The work storage unit 8 updates the intermediate state based on the operation signal sent from the selection unit 5 (402).
After updating the intermediate state, the work storage unit 8 sends the position and orientation of the robot 1 included in the updated intermediate state to the control unit 2 (403). The control unit 2 moves the robot 1 to the position and orientation of the robot 1 sent from the work storage unit 8 (404).

When the operation of the robot 1 is finished, the control unit 2
A signal is sent to the selection unit 5 (405). The selection unit 5 is the recognition unit 3
-2 is activated to confirm the operation of the robot 1. Here, the confirmation target is the operation target component of the robot 1, and in FIGS. 4 (c) to 4 (d), the component and FIG.
Parts (b) to 6 (c) are parts. The operation check of the robot 1 is performed as follows.

First, the recognition unit 3-2 uses the model of the confirmation target object stored in the model storage unit 9 and the predicted position and orientation of the confirmation target object stored in the working storage unit 8 as an intermediate state. And are used to recognize the image information from the camera 4 and confirm the operation target parts of the robot 1 (40
6). If the recognition unit 3-2 cannot confirm the confirmation target, the recognition unit 3-2 determines that an error has occurred during the assembly work and sends a signal to the selection unit 5 (40
7), the selection unit 5 stops the assembly work (40
8). When the recognition unit 3-2 can confirm the confirmation target object, the recognition unit 3-2 determines that the assembling work is correctly performed, and sends a signal to the selection unit 5 (409). The selection unit 5 sends a signal to the planning unit 7 (410), and the department store operation of the picking work is thus completed.

As described above, in the department store operation of the picking work in this embodiment, the operation of the robot 1 is confirmed according to the image recognition result by the recognition unit 3-2. As a result, when the robot 1 does not normally grasp the parts, the assembly work by the robot 1 is stopped as in the approach operation. The advantages of this outage mentioned in the description of the approach operation can be enjoyed as well and re-planning is possible as well.

(3.2D) When the operation signal indicates the approach operation for the place work The approach operation for the place work is shown in FIGS.
The operation is as shown in (a) or FIG. 6 (c) to FIG. 7 (a). This is an operation in which the hand of the robot 1 grasping a component approaches the component to a predetermined place. In this embodiment, the error correction of the intermediate state is executed in the approach operation of the place work.

As shown in FIG. 11, in this operation, the selection section 5 first sends an operation signal to the working storage section 8 (501). The work storage unit 8 updates the intermediate state to the state indicating the approached state based on the operation signal sent from the selection unit 5 (502). When the update of the intermediate state is completed, the work storage unit 8 stores the position and orientation of each component stored in the intermediate state and the model of each component stored in the model storage unit 9 into the support relation calculation unit 6 To obtain a supporting relationship between the respective parts after the approach (503). The work storage unit 8 is a component (contact component) that comes into contact with the operation target component of the robot 1 other than the table based on the support relationship between the components.
Search for (504). Here, although there are no contact parts in the examples of FIGS. 4 (d) to 5 (a), the parts correspond to these in the examples of FIGS. 6 (c) to 7 (a). If there are contact parts,
The work storage unit 8 activates the recognition unit 3-3 to correct the error in the intermediate state. Here, the object of correction is a contact component.

The error correction in the intermediate state is performed as follows.
First, the recognition unit 3-3 uses the model of the correction target object stored in the model storage unit 9 and the predicted position and orientation of the correction target object stored in the intermediate state of the work storage unit 8.
Is used to recognize the image information from the camera 4 and determine the position and orientation of the contact component that is the correction target (505).
If the recognition unit 3-3 cannot confirm the contact parts, the recognition unit 3-3 determines that an error has occurred during the assembly work and sends a signal to the selection unit 5 (506). 5 stops the assembling work (507). Recognition unit 3
If -3 can confirm the contact component, the recognition unit 3-3 sends the position and orientation of the contact component to the work storage unit 8 (508). The work storage unit 8 obtains an error between the position and orientation of the contact component sent from the recognition unit 3-3 and the predicted position and orientation of the contact component stored as the intermediate state (509). If there is an error, the work storage unit 8 corrects the position and orientation of the contact component stored as the intermediate state (510), and accordingly, the component that contacts the contact component stored as the intermediate state (robot). (Including the operation target component 1) is corrected.
In addition, the position and orientation of the robot 1 stored as the intermediate state is corrected along with the correction of the position and orientation of the operation target component.

When the error correction of the intermediate state is completed in this way, the work storage unit 8 sends the position and orientation of the robot 1 stored as the intermediate state to the control unit 2 (511).
The control unit 2 uses the robot 1 sent from the work storage unit 8.
Move the robot 1 so that
2). When the operation of the robot 1 ends, the control unit 2 controls the selection unit 5
(513). The selection unit 5 sends a signal to the planning unit 7 (514), and the approach operation of the place work is completed in this manner.

Therefore, in the approach operation of the place work in this embodiment, the error correction of the intermediate state is performed according to the image recognition result by the recognition unit 3-3. As a result, the robot 1 can accurately approach even if the contact component does not have the initially predicted position and orientation. Further, when the recognition unit 3-3 cannot confirm the contact component, the assembly work by the robot 1 is stopped, and the advantage of the operation stop described above can be enjoyed. Replanning is also possible.

(3.2E) When the operation signal indicates the release operation of the place work This operation is performed, for example, in FIGS. 5 (a) to 5 (b) and FIG.
This is the operation shown in (a) and (b). That is, this is an operation of opening the hand after approaching. This operation is shown in FIG.
2 is an operation in a flow as shown in FIG. 2, and unlike other operations, the recognition unit 3 is not activated.

In this operation, first, the selection unit 5 causes the control unit 2 to operate.
(601). The control unit 2 opens the hand of the robot 1 (602). When the operation of the robot 1 is finished, the control unit 2 sends a signal to the selection unit 5 (603). The selection unit 5 sends a signal to the planning unit 7 (604), and the release operation of the place work ends.

(3.2F) When the operation signal indicates the department store operation of the place work This operation is performed, for example, in FIG. 5 (b) to FIG. 5 (c) or FIG.
This is the operation shown in (b) to (c). That is, this is an operation of pulling up the hand of the robot 1 after releasing the parts. This operation is an operation characterized by the operation confirmation and the assembly confirmation of the robot. Further, in the case of this operation, the specific operation differs depending on whether or not there is a contact component in the surroundings.

As shown in FIG. 13, in this operation, the selection section 5 first sends an operation signal to the working storage section 8 (701). The work storage unit 8 updates the intermediate state based on the operation signal sent from the selection unit 5 (702).

After updating the intermediate state, the work storage unit 8
Sends the position and orientation of the robot 1 stored as the intermediate state to the control unit 2 (703). The control unit 2 moves the robot 1 to the position and orientation of the robot 1 sent from the work storage unit 8 (704). When the operation of the robot 1 ends, the control unit 2 sends a signal to the selection unit 5 (70
5). The selection unit 5 determines the presence / absence of a contact component based on the support relationship included in the intermediate state (706), and executes a different operation according to the result.

When there is no contact component For example, in the case of FIGS. 5B to 5C, there is no contact component, and the following operation is performed. First, in this case, the selection unit 5
Activates the recognition unit 3-2 and confirms the operation of the robot 1. The confirmation target is an operation target component of the robot 1.
Specifically, the operation confirmation of the robot 1 is as follows.

First, the recognition unit 3-2 recognizes the model of the confirmation target object stored in the model storage unit 9 and the predicted position and orientation of the confirmation target object stored in the intermediate state of the work storage unit 8. , Are used to recognize the image information from the camera 4 and confirm the confirmation target (707). When the recognition unit 3-2 cannot confirm the confirmation target object, the recognition unit 3-2 determines that an error has occurred during the assembly work, and sends a signal to the selection unit 5 (708) to select. The part 5 stops the assembling work (709). When the recognition unit 3-2 can confirm the confirmation target object, the recognition unit 3-2 determines that the assembling work is correctly performed, and selects the selection unit 5.
(710), and the selection unit 5 sends a signal to the planning unit 7 (711). In this way, the department store operation of the place work is completed.

When there is a contact component For example, in the case of FIGS. 7B to 7C, there is a contact component (component), and the following operation is performed. In this case, first, the selection unit 5 activates the recognition unit 3-3 to check the assembly of the robot 1. The objects to be confirmed are the operation target parts and the contact parts of the robot 1.

Assembly confirmation of the robot 1 is performed as follows. First, the recognition unit 3-3 sets the model of the confirmation target object stored in the model storage unit 9 and the predicted position and orientation of the confirmation target object stored in the work storage unit 8 as the intermediate state. By using the information, the image information from the camera 4 is recognized, and the position and orientation of the confirmation target object are obtained (71
2). The recognition unit 3-3 sends information to the support relationship calculation unit to obtain the support relationship between the operation target component and the contact component (713).
The selection unit 5 sets the support relationship between the confirmation objects sent from the support relationship calculation unit 6 and the support relationship between the confirmation objects in the target state stored in the work storage unit 8 as the target state support relationship. Compare (714). If the support relationship between the confirmation objects sent from the support relationship calculation unit 6 and the target state support relationship between the confirmation objects conflict, the selection unit 5 determines that an error has occurred during the assembly work. Then, the assembly work is stopped (715). If there is no contradiction between the support relationship between the confirmation objects sent from the support relationship calculation unit 6 and the target state support relationship between the confirmation objects, the selection unit 5 determines that the assembly work is correctly performed. Then, a signal is sent to the planning unit 7 (711), and the department store operation of the place work is completed.

As described above, in the department store operation of the place work, since the operation confirmation or the assembly confirmation of the robot 1 is performed, it is confirmed that the place work is performed correctly. Further, when the result of the confirmation shows that an error has occurred, the work is stopped, so that it is possible to easily deal with it and re-plan is possible.

[0069]

As described above, according to the present invention,
Image recognition is performed at a predetermined point during assembly work, while
Since the intermediate state indicating the desired position of the parts etc. is sequentially updated, and both are collated, the position and / or posture of the robot is corrected and controlled according to the plan so that the predetermined state can be obtained. It becomes possible to correct the error and deal with the error immediately after these occurrences.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an assembly device using a robot according to an exemplary embodiment of the present invention.

FIG. 2 is a flow chart showing a flow of operation up to generation of an operation signal in this embodiment.

FIG. 3 is a diagram for explaining the operation of this embodiment,
3A is a perspective view showing an initial state, and FIG. 3B is a perspective view showing a target state.

4A and 4B are diagrams showing the flow of a picking operation executed from the initial state shown in FIG. 3A, FIG. 4A being before the approach operation, FIG. 4B being after the approach operation, Figure 4
FIG. 4C is a perspective view showing a state after the grasp operation and FIG. 4D is a state after the department store operation.

FIG. 5 is a diagram showing a flow of a place work following FIG. 4 (d), and FIG. 5 (a) is a diagram after FIG. 5 (b) after the approach operation.
Is after the release operation, and FIG. 5 (c) is after the department store operation.
It is a perspective view showing a state.

FIG. 6 is a diagram showing the flow of the picking work following FIG. 5 (c), FIG. 6 (a) is after the approach motion, FIG. 6 (b) is after the grasp motion, and FIG. 6 (c) is the department store. It is a perspective view showing a state after the operation.

FIG. 7 is a diagram showing a flow of a place work following FIG. 6 (c), and FIG. 7 (a) is a diagram after FIG. 7 (b) after the approach operation.
Is after the release operation, and FIG. 7 (c) is after the department store operation,
It is a perspective view showing a state.

FIG. 8 is a flowchart showing a flow of approach operation of a pick work in the present embodiment.

FIG. 9 is a flowchart showing a flow of a grasping operation of a picking operation in this embodiment.

FIG. 10 is a flowchart showing a flow of a department store operation of pick work in the present embodiment.

FIG. 11 is a flowchart showing the flow of approach operation for place work in the present embodiment.

FIG. 12 is a flowchart showing a flow of a place work release operation in the present embodiment.

FIG. 13 is a flowchart showing the flow of a department store operation of a place work in this embodiment.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Robot 2 Control part 3,3-1,3-2,3-3 Recognition part 4 Camera 5 Selection part 6 Supporting relation calculation part 7 Planning part 8 Working memory part 9 Model memory part 10 Input part

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] October 3, 1991

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0008

[Correction method] Change

[Correction content]

On the other hand, in the present invention, the motion of the robot is
Work type, robot movement amount, robot operation target parts
Based on the above, the information related to the intermediate state is sequentially updated. The intermediate state thus obtained indicates the desired arrangement of the robot and each component.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0011

[Correction method] Change

[Correction content]

(1) Construction of Embodiment FIG. 1 shows the construction of a robot assembly apparatus according to an embodiment of the present invention. It is the robot 1 that performs the assembly work with the apparatus shown in this figure. The operation of the robot 1 is controlled by the control unit 2. The recognition unit 3 has a function of recognizing a component or a hand of the robot 1, and includes recognition units 3-1, 3-2 and 3-3 as described later. It is the camera 4 that captures the image information of the part or the hand of the robot 1 and supplies it to the recognition unit 3. Further, the selection unit 5 performs a necessary operation according to the supplied signal, the support relationship calculation unit 6 calculates the support relationship between the parts, and the planning unit 7 plans the operation sequence of the robot 1. The work storage unit 8 is a means for storing an initial state and the like, and the model storage unit 9 is a means for storing a model of each component and the hand of the robot 1 . The input unit 10 is means for a person to input these pieces of information into the work storage unit 8 and the model storage unit 9 from the outside.

[Procedure 3]

[Document name to be amended] Statement

[Name of item to be corrected] 0019

[Correction method] Change

[Correction content]

The recognition unit 3-2 also recognizes the recognition target objects (parts and the robot 1 handle stored in the model storage unit 9).
And models de), by using the predicted position and orientation of the recognition object is stored as an intermediate state in the working memory 8 (hand part and the robot 1), a predetermined range including the position and orientation Object to be recognized in (part and robot 1
(Hands) actually exists. this is,
It is used to confirm the operation of the robot 1.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0020

[Correction method] Change

[Correction content]

The recognition unit 3-3 is then connected to the model storage unit 9
The recognition target (parts and robot 1
Model ) and the recognition object (part and hand of the robot 1) stored as an intermediate state in the work storage unit 8.
Using the predicted position and orientation of the object, the recognition target (parts and robot
After confirming that the first hand) actually exists, the actual position and orientation of the recognition object (the part and the hand of the robot 1) are obtained. This is the confirmation of the assembly of the robot 1.
It is used for the error correction of the robot 1 and the error correction of the intermediate state.

[Procedure Amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0021

[Correction method] Change

[Correction content]

The recognition units 3-2 and 3-3 are objects to be recognized.
In the case where the (parts and the hand of the robot 1) cannot be confirmed, in this embodiment, it is determined that an error has occurred during the assembly work, and the work is stopped. Also, the recognition unit 3
-3 required recognition object (hands of parts and robot 1
The actual position and orientation and recognition object de) (Parts & B
If there is an error between the predicted position and orientation of the hand of the bot 1) , it is determined that an error has occurred during the assembly work, and the error is corrected.

[Procedure correction 6]

[Document name to be amended] Statement

[Name of item to be corrected] 0023

[Correction method] Change

[Correction content]

(3.1) From Input to Output of Operation Signal FIG. 2 shows a flow of operation from input to output of operation signal in this embodiment. In this embodiment, first, the user needs to input necessary information from the input unit 10 and store it in the work storage unit 8 and the model storage unit 9 (101). The information to be input includes the initial state, the target state, the initial state support relationship, the target state support relationship, the work environment knowledge, each part, and the model of the hand of the robot 1 .
As described above, the last one of these is stored in the work storage unit 8, and the model of each component and the hand of the robot 1 is stored in the model storage unit 9. However, regarding the initial state, the target state, the initial state support relationship, and the target state support relationship, the recognition unit 3-1 or the support relationship calculation unit 6 can be used, and it is not always necessary to input from the input unit 10.
When the input is completed, the input unit 10 sends a signal to the planning unit 7 (102).

[Procedure Amendment 7]

[Document name to be amended] Statement

[Correction target item name] 0038

[Correction method] Change

[Correction content]

As described above, in each operation of the assembly work performed by using the apparatus of this embodiment, the operation check of the robot 1, the assembly check of the robot 1, the error correction of the robot 1, the intermediate state Characteristic operations such as error correction are executed. In these operations, the recognition unit 3 operates, and the robot stands still at the break of a series of operations.
Image recognition is performed on the hand or the part. Next, details of the basic operation of the present embodiment will be described, and the characteristic operations thereof will be described.

[Procedure Amendment 8]

[Document name to be amended] Statement

[Item name to be corrected] 0040

[Correction method] Change

[Correction content]

First, the selection unit 5 outputs the operation signal to the work storage unit 8
To (201). The work storage unit 8 updates the intermediate state based on the operation signal sent from the selection unit 5 (20
2). In this step 202, more specifically,
The work storage unit 8 makes the robot in the intermediate state based on the operation signal.
The position and posture of the robot 1 are updated, and the position and posture of the robot 1 are updated.
The position and posture of each part is measured from the posture and the motion of the robot 1.
Then, the position and orientation of each component in the intermediate state are updated.

[Procedure Amendment 9]

[Document name to be amended] Statement

[Name of item to be corrected] 0045

[Correction method] Change

[Correction content]

(3.2B) When the motion signal indicates the grasp motion of the picking work This motion is the motion shown in FIGS. 4 (b) to 4 (c) and 6 (a) to 6 (b). and is a operation for grasping the part that will be executed after the approach operation. In this operation, the selection unit 5 first activates the recognition unit 3-3 to correct the error of the robot 1.

[Procedure Amendment 10]

[Document name to be amended] Statement

[Correction target item name] 0052

[Correction method] Change

[Correction content]

First, the recognition unit 3-2 uses the model of the confirmation target object stored in the model storage unit 9 and the predicted position and orientation of the confirmation target object stored in the working storage unit 8 as an intermediate state. Using and, the image information from the camera 4 is recognized and the confirmation target is confirmed (406). Recognition unit 3
-2 is unable to confirm the confirmation target, the recognition unit 3-2 determines that an error has occurred during the assembling work and sends a signal to the selection unit 5 (407).
Stops the assembly operation (408). Recognition unit 3-2
Is able to confirm the confirmation target, the recognition unit 3
-2 judges that the assembling work is performed correctly, and sends a signal to the selecting unit 5 (409). The selection unit 5 sends a signal to the planning unit 7 (410), and the department store operation of the picking work is thus completed.

[Procedure Amendment 11]

[Document name to be amended] Statement

[Correction target item name] 0056

[Correction method] Change

[Correction content]

The error correction in the intermediate state is performed as follows.
First, the recognition unit 3-3 uses the model of the correction target object stored in the model storage unit 9 and the predicted position and orientation of the correction target object stored in the intermediate state of the work storage unit 8.
Is used to recognize the image information from the camera 4 and determine the position and orientation of the contact component that is the correction target (505).
If the recognition unit 3-3 cannot confirm the contact parts, the recognition unit 3-3 determines that an error has occurred during the assembly work and sends a signal to the selection unit 5 (506). 5 stops the assembling work (507). Recognition unit 3
If -3 can confirm the contact component, the recognition unit 3-3 sends the position and orientation of the contact component to the work storage unit 8 (508). The work storage unit 8 obtains an error between the position and orientation of the contact component sent from the recognition unit 3-3 and the predicted position and orientation of the contact component stored as the intermediate state (509). If there is an error, working memory unit 8 corrects the position and orientation of the contact parts stored as an intermediate state, with the Re their parts in contact with the contact part being stored as the intermediate state (the robot 1 (Including the operation target component of 1)) is corrected. In addition, the position and orientation of the robot 1 stored as an intermediate state is corrected along with the correction of the position and orientation of the operation target component.
(510) .

[Procedure Amendment 12]

[Document name to be corrected] Drawing

[Name of item to be corrected] Fig. 4

[Correction method] Change

[Correction content]

[Figure 4]

Front page continuation (72) Inventor Tadashi Asaoka 2-18 Keihan Hondori, Moriguchi City, Osaka Sanyo Electric Co., Ltd. (72) Inventor Hajime Terasaki 2-18 Keihan Hondori, Moriguchi City, Osaka Sanyo Electric Co., Ltd. (72) Inventor Hironobu Takahashi 2-18 Keihan Hondori, Moriguchi City, Osaka Sanyo Electric Co., Ltd. (72) Inventor Kazuhide Sugimoto 2-18 Keihan Hondori, Moriguchi City, Osaka Sanyo Electric Co., Ltd.

Claims (1)

Claim: What is claimed is: 1. A robot capable of moving a plurality of parts to be assembled, a means for recognizing an image of the position and / or posture of the robot and / or the parts at a stationary point during an assembly operation, and a robot. Means for planning the operation sequence of, at least the initial state of the arrangement of the parts, the target state that is a state after the parts are assembled, the support relationship between the parts in the initial state, the support relationship between the parts in the target state,
Means for storing an intermediate state indicating a desired layout of the parts at a predetermined time point during the assembly work, information regarding the robot and a model which is the shape of the parts, and information regarding the intermediate state based on the stored information. And the means for correcting and controlling the position and / or the posture of the robot according to the plan so as to be in a predetermined state while collating the information related to the intermediate state with the result of the image recognition. An assembling apparatus for a robot, which performs an assembling operation while monitoring the state of the robot and the operation result of the robot.