JP2003231078A - Position control method for robot arm and robot device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To normally fit a work to a fitting object by correcting its deviation even if the grasped work is deviated. <P>SOLUTION: This robot device grasps the work 2 by a robot arm 3, moves the robot arm 3 to a position right before fitting the work 2 to the fitting object 4, photographs the work 2 by a CCD camera 16, stores it as a comparison object image, and stores the position of the robot arm 3, which is moved for fitting the work 2 to the fitting object 4, as a target position by relating it to the comparison object image. In a practical work, this robot device grasps the work 2 by the robot arm 3 whose position attitude is controlled by a visual sense feedback control, photographs the grasped work 2 by the CCD camera 16, compares the photographed image with multiple comparison object images, searches the comparison object image according with or adjoining to the photographed image, and moves the robot arm 3 to a target position corresponding to the searched comparison object image. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot arm position control method and a robot apparatus for feedback-controlling the position and orientation of the robot arm on the basis of data of an image picked up by a visual sensor provided on the robot arm. Is.

[0002]

For example, in the control technology of an assembling robot, when a work is gripped by a robot arm, the work is photographed by a camera attached to the robot arm to control the position and orientation of the robot arm. There is a technology of such visual feedback control (so-called visual servo). This is because, for example, when teaching, the robot arm is moved to a position immediately before gripping the work, the work is photographed by the camera attached to the robot arm at that position, and the image is stored in advance as a reference image. , The work is always photographed by the camera, the photographed image and the reference image are compared, and the position and orientation of the robot arm are controlled so that the error between them is zero, that is, the appearance is the same as when teaching. According to this technique, the position and orientation of the hand can be accurately and promptly controlled.

If such a visual feedback control technique is adopted, the position control when the hand grips the work should be accurately performed. However, if it is adopted in the actual assembling work, it will be as follows. The inventors' research has revealed that a problem occurs.

For example, when the hand grips the work, the work sometimes shifts. In particular, in the case of a hand configured to grip a work by vacuum suction, it often happens that the work moves when it is sucked. If the work shifts with respect to the hand, then if the robot arm is moved to the assembly position and the work is to be assembled on the other side,
The work may be displaced from the regular assembly position due to the displacement with respect to the hand, and the assembly may become impossible.

The present invention is intended to solve the problem that occurs in such visual feedback control, and an object thereof is to correct the deviation even if the gripped work is displaced and to pass it to the other party normally. (EN) Provided is a robot arm position control method and a robot apparatus capable of performing the same.

[0006]

In order to achieve the above-mentioned object, the invention of claim 1 holds an object by a robot arm whose position and orientation are controlled by visual feedback control before actual work, and thereafter, A visual sensor that attaches the object gripped by the robot arm to the robot arm by repeatedly performing the teaching operation in which the robot arm is moved to a position for passing the object to the other side, for each teaching operation. And store this as a comparison target image, and store the position of the robot arm that was moved to pass the target object to the other side as a target position in association with the comparison target image. The process of gripping an object with a robot arm whose position and orientation is controlled by a visual sensor , The captured image is compared with a plurality of comparison target images stored in the storage unit to search for a comparison target image that matches or is close to the captured image, and a target position corresponding to the searched comparison target image is acquired. Process and
The process of moving the robot arm to the acquired target position is sequentially executed.

According to the means of claim 1, the teaching operation is performed several times and a large number of comparison target images are stored, so that when the target object is gripped in the actual work, the target object becomes normal. Regardless of whether or not there is a shift from the grip position, regardless of the amount of shift when there is a shift, there is a comparison target image that matches or is close to the image of the target image captured in that gripping state. By moving the robot arm to the same target position, the gripped target object can be passed normally to the other party. Further, since the displacement is not detected by the visual sensor and the displacement is corrected, the troublesome calibration of converting the coordinates on the image of the visual sensor into the robot coordinates to obtain the displacement is unnecessary.

According to a second aspect of the present invention, an object is grasped by a robot arm whose position and orientation are controlled by visual feedback control, and then the robot arm is moved to a position for passing the object to the other party. When performing a teaching that repeats the action to be performed a plurality of times, for each action, the object grasped by the robot arm is photographed by the visual sensor attached to the robot arm and this is stored as a comparison target image, and the object is also compared. A storage unit that stores the position of the robot arm that has been moved to the other side as a target position in association with the comparison target image, and at the time of actual work, grips the target object with the robot arm whose position and orientation are controlled by visual feedback control. Then, the grasped object was photographed by the visual sensor, and the photographed image was stored in the storage means. Search means for searching a comparison target image that matches or is close to the captured image by comparing the number of comparison target images, and acquires a target position corresponding to the searched comparison target image; and a target position acquired by the search means. And a control means for moving the robot arm. The same effect as that of the first aspect can be obtained by the means of the second aspect.

According to the third aspect of the invention, prior to the actual work, the robot arm holding the object moves the object to a position where the visual sensor for inspection installed at a fixed position can photograph the object, and at this position, the object for inspection is inspected. After shooting the object with the visual sensor,
Teaching is performed to move the robot arm to a position for passing the object to the other side, and at that time, the image taken by the inspection visual sensor is stored as an inspection image, and the position of the robot arm at the time of this imaging and The position of the robot arm moved to pass the object to the other side is stored as the inspection reference position and the target reference position, respectively, and the actual work is grasped by the robot arm whose position and orientation are controlled by visual feedback control. After that, the robot arm is moved to a position where the inspection visual sensor can image the object, and the position of the robot arm is controlled so that the image captured by the inspection visual sensor matches the inspection image stored in the storage means. The process of detecting the current position when
The present invention is characterized in that an actual target position is obtained from the detected current position, inspection reference position, and target reference position, and the process of moving the robot arm to this actual target position is sequentially performed.

According to the means of claim 3, in the actual work, the robot arm is moved so that the image of the object photographed by the visual inspection sensor coincides with the inspection image at the time of teaching. As a result, the robot arm is moved to a position that has moved from the inspection reference position at the time of teaching by the amount corresponding to the deviation of the work. Then, at the time of teaching, by moving the robot arm from the inspection reference position to the target reference position,
Since the object was successfully delivered to the other side, the target is moved by moving the robot arm from the current position by an amount (vector) that is the amount obtained by subtracting the inspection reference position from the reference reference position and the deviation amount. You can pass the item to the other party normally. In addition, the deviation is automatically corrected by moving the robot arm so that the image captured by the inspection visual sensor matches the inspection image, so calibration that converts the coordinates of the inspection visual sensor into robot coordinates is not required. Is.

According to a fourth aspect of the present invention, the inspection visual sensor installed at a fixed position and the robot arm holding the object are moved to a position where the inspection visual sensor can photograph the object prior to actual work. Then, after shooting the object with the inspection visual sensor at this position, when the instruction to move the robot arm to the position for passing the object to the other side is executed, the image taken with the inspection visual sensor is taken as the inspection image. And a storage means for storing the position of the robot arm at the time of this photographing and the position of the robot arm moved to pass the object to the other side as the inspection reference position and the target reference position, respectively, and at the time of actual work, After grasping the target object by the robot arm whose position and orientation are controlled by visual feedback control, the robot arm inspects the target object by the visual sensor for inspection. Position detecting means for detecting the current position when the position of the robot arm is controlled so that the image picked up by the visual inspection sensor coincides with the inspection image stored in the storing means by moving to a possible position, and this position The calculation means for obtaining the actual target position from the current position, the inspection reference position and the target reference position detected by the detection means, and the control means for moving the robot arm to the actual target position obtained by the calculation means are provided. Characterize. The same effect as in claim 3 can also be obtained by the means of claim 4.

[0012]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment in which the present invention is applied to a vertical articulated robot apparatus for assembly will be described with reference to FIGS.
Will be described with reference to. FIG. 1 schematically shows the configuration of an assembly system using the robot apparatus 1. In this assembly system, a robot arm 3 grips a work 2 as an object supplied on a work table (not shown) at an arbitrary position and posture, and the work 2 is conveyed to an assembly station by a conveyor (not shown). It is to be attached to the attached body 4. Although not shown, a plurality of feature points (specific edges or holes, affixed markers, etc.) are set on the work 2 for visual recognition described later.

The robot arm 3 constitutes a robot body. This robot arm 3 is composed of a base portion 5, a shoulder portion 6, a lower arm portion 7, an intermediate arm portion 8, an upper arm portion 9, a wrist portion 10, a flange portion 11 and the like, and the connection configuration of these respective portions is as follows. ing. That is, the base portion 5 is fixed to the installation floor surface, and the shoulder portion 6 rotates (turns) around the vertical axis on the base portion 5.
A pair of lower arm portions 7 are connected to the left and right ends of the shoulder portion 6 so as to be vertically rotatable about a horizontal axis. Between the tips of the pair of lower arm portions 7,
A front end portion of the intermediate arm portion 8 is connected so as to be vertically rotatable about a horizontal axis.

The rear end of the upper arm 9 is coaxially rotatably connected to the front end of the intermediate arm 8, and the wrist 10 is formed at the left and right ends of the upper arm 9.
Is rotatably connected about a horizontal axis, and its wrist portion 1
A flange portion 11 is coaxially rotatably connected to the front end portion of 0. Then, the flange 12 has a hand 12
(Illustrated in FIG. 5) is detachably attached.

The respective parts 6 to 11 of the robot arm 1 described above
Are driven by servomotors (not shown) with encoders, and these servomotors are controlled by a robot controller 14 forming a part of the controller 13. .

The controller 13 is a robot controller 1
4, a visual recognition device 15, CC as a visual sensor
The D camera 16 is provided. Visual recognition device 1
5 and the CCD camera 16 are for visual feedback control, and the CCD camera 16 is attached to, for example, the wrist unit 10 among the joints of the robot arm 3.

The robot controller 14 is mainly composed of a CPU, is provided with a ROM and a RAM for storing a control program, and is composed of a servo control section for outputting a control command as a drive signal to the robot arm 3.
Further, the visual recognition device 15 is mainly provided with a CPU, and is provided with a ROM and a RAM that store a program for position and orientation control, and is also provided with an interface circuit for inputting an image signal from the CCD camera 16, an image processing circuit, an image memory, and the like. It has a different structure.

The visual recognition device 15 sends a control operation signal (position correction amount) to the robot controller 1 based on image processing.
It is supposed to be given to 4. In this case, in the visual recognition device 15, the registration data of the reference image obtained by the teaching operation executed in advance is stored in the image memory. This teaching operation is, for example, a teaching pendant (not shown).
The robot arm 3 is moved to a teaching target position (for example, a position and posture immediately before gripping the work 2 with a hand) with respect to the work 2 by a manual operation using, and the work 2 is photographed by the CCD camera 16 at this position. The captured image is registered (stored) in the visual recognition device 15 as a reference image.

FIG. 2 shows registration data of a reference image obtained by photographing the work 2 with the CCD camera 16. The registration data is set with the positional relationship between the reference point and the characteristic point in the field of view of the CCD camera 16 as the characteristic amount. That is, an arbitrary reference point P0 is set on the visual field, and the work 2
The characteristic points on the shape of are defined as characteristic points P1 to P5. Then, on the photographed image, vectors V1 to V5 formed by connecting the reference point P0 and the respective characteristic points P1 to P5
And the data of the angles φ1 to φ5 formed by the adjacent vectors V1 to V5 are calculated, and this feature amount data is used as the registration data of the reference image.

During actual work, the control device 1
Reference numeral 3 denotes an image of the work 2 taken by the CCD camera 16 at an arbitrary position and orientation, and the picked-up image data is processed by the visual recognition device 15 to process the feature amount (data of V1 to V5, angles φ1 to φ5) on the current image. ) Is similarly obtained as the current data (feedback signal), and the current data is compared with the registered data of the reference image to reduce the error (deviation) (for example, to 0), that is, the CCD camera 16
The position correction amount is calculated so that the image according to the above-described image looks the same as at the time of teaching and is output to the robot controller 14 as a control operation signal. The position correction amount is converted into a velocity vector that gives the operating speed of the flange portion 11 of the robot arm 3 and given to the robot controller 14.

The robot controller 14 repeatedly controls the position and orientation of the robot arm 3 on the basis of the given position correction amount, and when the error is converged to 0 or becomes a predetermined value or less by this control, that is, the robot arm. When 3 reaches the teaching target position, the gripping operation by the hand 12 is started. Therefore,
The visual recognition device 15 functions as a storage unit for the reference image,
The robot controller 14 functions as control means for controlling the operation of the robot hand 3.

In the present embodiment, when the work 2 is gripped by the hand 12, even if the work 2 is misaligned, the work 2 can be assembled to the body 4 to be assembled. I try to do the following: That is, in the teaching process, first, in order to register the above-mentioned reference image, the robot arm 3 is moved to the teaching target position by the manual operation of the teaching pendant as shown in the flowchart of teaching 1 in FIG. Step A1), the work 2 is photographed at this teaching target position (step A2), and the feature amount of the work 2 on the image at that time is stored in the image recognition device 15 as reference image registration data (step A2).

After that, the robot arm 3 is moved to the target gripping position by visual feedback control,
As shown in (a), the work 2 is grasped by the hand 12, and then manually operated to a position immediately before the work 2 is assembled to the assembly body 4 (for example, when the work 2 is lowered directly below, the work 2 is assembled). The robot arm 3 is moved to a position where it can be attached to the body 4, and as shown in FIG. 5B, the teaching operation of assembling the work 2 to the body 4 to be attached is performed an arbitrary number of times (n). Execute repeatedly.

Then, as shown in the flowchart of Teaching 2 in FIG. 3B, after each teaching operation, the robot arm 3 is moved to the target gripping position and the work 12 is gripped by the hand 12, and then the work 2 is CCD. The image is taken by the camera 16 (steps B1, 2, and 3), and the feature amount on the image is stored as a comparison target image in the visual recognition device 15 serving as a storage unit (step B4). Then, the robot arm 3 is moved so that the work 2 comes to a position immediately before the work 4 is mounted on the assembly target 4, and the position of the robot arm 3 is stored as a target position in association with the feature amount data of the comparison target image ( Above, steps B5, 6). The position of the robot arm 3 at this time is stored as a position on the robot coordinates. The position on the robot coordinates is based on the output of the encoder of the servo motor that operates each part of the robot arm 3. Is obtained by calculating. Therefore, the robot controller 14 functions as a current position detecting means for detecting the current position of the robot arm 3.

By performing the teaching operation a plurality of times in this way, various gripping states can be obtained from the gripping state in which the work 2 does not move with respect to the hand 12 to various normally considered slipping movements. It is possible to acquire the movement position of the robot arm 3 that can normally assemble the workpiece 2 to the assembly target 4 for each of the various displacement movements.

During actual work, the hand 12 is controlled by performing the control shown in the flowchart of FIG.
Even if the work 2 gripped by the robot shifts, the work 2 can be normally assembled to the mounted body 4. The actual work is
It starts from the movement of the robot arm 3 to the gripping target position by the visual feedback control (step C
1) Then, the work 2 is gripped by the hand 12 (step C2). Then, in the gripping state of the work 2, the work 2 is photographed by the CCD camera 16 (step C3). The image captured by the CCD camera 16 is input to the visual recognition device 15, and the visual recognition device 15 receives the image.
Calculates the feature amount on the image of the input work 2, compares the feature amount with the feature amounts of the plurality of comparison target images stored in the image memory (step C4), and matches or approaches (approximates). The image is searched, and the target position associated with the searched comparison target image is searched (search means: step C5).

After that, the visual recognition device 15 outputs the retrieved target position data to the robot controller 14,
The robot controller 14 controls the robot arm 3 so as to move it from the current position toward the target position (step C6). By this control, the work 2 is set to a position where it can be attached to the assembly body 4, and thereafter, the work 2 is moved to a position just below by a predetermined amount, whereby the work 2 is assembled. 4 is assembled (step C7). The above operation is continued until the work 2 disappears (“YES” in step C8) or the stop operation is performed (“YES” in step C9).

In this embodiment, the hand 12 is used during actual work.
Taking the image of the work 2 gripped by the CCD camera 16 and selecting an image that is the same as or close to the taken image from among a large number of comparison target screens means that the work during actual work is selected from a large number of comparison target images. This is to select the work 2 having the same deviation amount as 2. Therefore, according to the present embodiment, even during actual work, if the robot arm 3 is moved to the same position as the target position of the work 2 on the selected comparison target screen,
The work 2 can be normally attached to the attached body 4.

In contrast to this embodiment, the work 2 gripped by the hand 12 is photographed by the CCD camera 16 and the difference from the regular gripping position on the photographed screen is obtained as the deviation amount, and only the deviation amount is obtained. A configuration in which the moving position of the robot arm 3 is corrected is conceivable, but in this case, C
It is necessary to convert the coordinates of the image captured by the CD camera 16 into robot coordinates (calibration), which makes teaching very troublesome. However, in the present embodiment, such calibration is not necessary and teaching work is performed. Will also be easier.

6 to 9 show another embodiment of the present invention. The hardware difference from the above-mentioned one embodiment is that the work 2
The inspection CCD camera 17 as an inspection visual sensor is installed at a predetermined position between the position for gripping the workpiece and the position for assembling the workpiece 2 to the assembly body 4. In the following description, the same parts as those in the above-described embodiment are designated by the same reference numerals, detailed description thereof will be omitted, and only different parts will be described.

As shown in the flowchart of FIG. 7, the work 2 is gripped by the hand 12 (steps D1, D
2) By manually operating the teaching pendant, the robot arm 3 is moved to a position where the workpiece 2 is photographed by the inspection CCD camera 17 (step D3).
The position of the robot arm 3 is stored in the visual recognition device 15 as the inspection reference position (P1) (step D4), and the workpiece 2 is photographed by the inspection CCD camera 17 at that position (step D5). The feature amount on the image is stored in the visual recognition device 15 as an inspection image (step D6). After that, by manual operation,
The position immediately before the work 2 is attached to the attached body 4 (for example,
(Position where it can be attached to the assembly 4 by lowering it directly)
Move the robot arm 3 so that it comes to (Step D
7) Then, the position of the robot arm 3 at this time is stored in the image memory as the target reference position (P2) (step D8).

At the time of actual work, by performing the control shown in FIG. 8, even if the work 2 gripped by the hand 12 shifts, it can be accurately assembled to the assembly body 4. There is. That is, the robot arm 3 is moved to the teaching target position by visual feedback control (step E1), and the work 2 is gripped by the hand 12 (step E2).

After that, the robot arm 3 is moved to the inspection reference position P1 (step E3), and the inspection C is made at this position.
The work 2 is photographed by the CD camera 17 (step E4). The feature amount of the captured image is compared with the stored feature amount of the inspection image, and if there is an error, the robot arm 3 is operated so that the error becomes 0 (step E5). Then, the position of the robot arm 3 when the error becomes 0 is obtained as the current position P3 (step E
6). The current position at this time is also the robot controller 14
Is obtained by calculating based on the output of the encoder of the servo motor that operates each part of the robot arm 3. Therefore, the robot controller 14 functions as a current position detecting means. The difference between the present position P3 and the inspection reference position P1 stored in the image memory is the deviation amount ΔP with respect to the grip position of the work 2 during teaching.

Next, a position for moving the robot arm 3 is obtained from the current position P3, the inspection reference position P1 and the target reference position P2. Specifically, a position (P3 + (P2-P1)) obtained by adding the value obtained by subtracting the inspection reference position P1 from the target reference position P2 to the current position P3 is obtained, and the actual target position P4 for moving this position from the current position P3. (Calculation means: step E7). Next, the robot arm 3 is moved to this actual target position P4 (step E8). This position P4 is a position where the deviation of the work 2 is corrected and the work 2 is moved to a position where the work 2 can be normally assembled to the assembly target object 4. Therefore, the robot arm 3 is subsequently operated to move the work. 2 is attached to the attached body 4 (step E
9). The above operation is continued until the work 2 disappears (“YES” in step E10) or a stop operation is performed (“YES” in step E11).

As described above, according to this embodiment, the inspection CC
The work 2 is photographed by the D camera 17, and the work 2
The robot arm 3 is operated so as to be at the same position as on the captured image at the time of teaching, and from this position (P3) at the same distance and direction as when the robot arm 3 is moved from the inspection reference position to the target reference position at the time of teaching. Since it is moved, the positional deviation of the work 2 with respect to the hand 12 can be corrected and the work 4 can be normally assembled to the assembly target 4.

Moreover, the deviation is automatically corrected by moving the robot arm so that the image taken by the inspection CCD camera 17 coincides with the image for inspection. Therefore, it is not necessary to obtain the amount of deviation on the taken image. Calibration for converting the coordinates of the inspection CCD camera 17 into robot coordinates is unnecessary.

The present invention is not limited to the embodiments described above and shown in the drawings, and the following expansions and modifications are possible. The target position in one embodiment, the inspection reference position and the target reference position in the other embodiments may be stored in the memory of the robot controller 13.
In one embodiment, the target position may be stored as an image, and the robot arm 3 may be moved to the target position by visual feedback control using this image as a reference image. The inspection CCD camera 16 is not limited to the wrist unit 10 and may be attached to the upper arm unit 9 or may be attached to another joint of the robot arm 3. The form of the robot arm is not limited to the vertical articulated type. The hand 12 is not limited to the vacuum suction type. The present invention is not limited to the case where the work 2 is assembled to the body 4 to be assembled, but can be widely applied to the case where the object is passed to the opponent side, and the opponent side may be a work table on which the object is placed.
It may be a pallet or the like.

[Brief description of drawings]

FIG. 1 is an overall schematic configuration diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory diagram of a storage form of a captured image of a work.

FIG. 3 is a flowchart of a teaching operation

[Fig. 4] Flow chart of actual work

FIG. 5 is an explanatory diagram of a teaching operation.

FIG. 6 is a view corresponding to FIG. 1 showing another embodiment of the present invention.

FIG. 7 is a view corresponding to FIG.

FIG. 8 is a view corresponding to FIG.

FIG. 9 is an explanatory diagram of an operation mode during teaching and actual work.

[Explanation of symbols]

In the figure, 1 is a robot device, 2 is a work (object), 4 is a robot arm, 12 is a hand, 14 is a robot controller (control means, current position detection means, calculation means), 1
Reference numeral 5 is a visual recognition device (storage means), 16 is a CCD camera (visual sensor), and 17 is an inspection CCD camera (inspection visual sensor).

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Takashi Naito             Aichi Prefecture Nagachite Town Aichi District             Ground 1 Toyota Central Research Institute Co., Ltd. (72) Inventor Hideki Nomura             Aichi Prefecture Nagachite Town Aichi District             Ground 1 Toyota Central Research Institute Co., Ltd. F term (reference) 3C007 BS10 BS12 BT08 KS21 KT01                       KT05 KT06 KV01 LT06 LT12                       MT04                 5H269 AB21 AB33 JJ09 SA08

Claims (4)

[Claims] 1. An image of an object such as a work is previously photographed by a visual sensor attached to a robot arm and stored as a reference image, and the image of the object photographed by the visual sensor is the same as the reference image. A robot device for performing visual feedback control for controlling the position and orientation of the robot arm so that the target object is controlled by the robot arm whose position and orientation is controlled by the visual feedback control prior to actual work. A teaching operation of gripping an object and then moving the robot arm to a position for passing the object to the other side is repeatedly executed a plurality of times, and the robot arm is gripped by each teaching operation. The object is photographed by the visual sensor and stored as a comparison target image, and the object is also captured. The position of the robot arm moved to be passed to the other party is stored as a target position in association with the comparison target image, and the actual work is performed by the robot arm whose position and orientation are controlled by the visual feedback control. And a process of photographing the grasped object with the visual sensor, and comparing the photographed image with the plurality of comparison target images stored in the storage means to match or approach the photographed image. A robot arm characterized by performing a process of searching a target image and acquiring a target position corresponding to the searched comparison target image, and a process of moving the robot arm to the acquired target position in order. Position control method. 2. An image of an object such as a work is previously photographed by a visual sensor attached to a robot arm and stored as a reference image, and the image of the object photographed by the visual sensor is the same as the reference image. A robot device for performing visual feedback control for controlling the position and orientation of the robot arm so as to be in a state, wherein the object is grasped by the robot arm whose position and orientation is controlled by the visual feedback control, After that, when performing a teaching in which the operation of moving the robot arm to move the object to a position for passing the object to the other side is repeated a plurality of times, the object grasped by the robot arm is moved for each operation. The image is taken by the visual sensor and stored as a comparison target image, and the target object is passed to the other party. Storage means for storing the position of the robot arm moved for the purpose as a target position in association with the comparison target image, and at the time of actual work, the object is controlled by the robot arm whose position and orientation are controlled by the visual feedback control. The object to be grasped is photographed by the visual sensor, the photographed image is compared with the plurality of comparison images stored in the storage unit, and a comparison image matching or close to the photographed image is displayed. A robot apparatus comprising: a searching unit that searches and acquires a target position corresponding to the searched comparison target image; and a control unit that moves the robot arm to the target position acquired by the searching unit. 3. An image of an object such as a work is previously photographed by a visual sensor attached to a robot arm and stored as a reference image, and the image of the object photographed by the visual sensor is the same as the reference image. A robot device for performing visual feedback control for controlling the position and orientation of the robot arm so that the robot arm holding the target object is inspected at a fixed position prior to actual work. A visual sensor for moving the object to a position where the object can be photographed, the visual sensor for inspection photographs the object at this position, and then the robot arm is moved to a position for passing the object to the other side. Is executed and, at that time, the image captured by the inspection visual sensor is stored as an inspection image, and the robot arm at the time of this imaging is stored. And the position of the robot arm moved to pass the object to the other side are stored as an inspection reference position and a target reference position, respectively, and the actual work is controlled by the visual feedback control to control the position and orientation. After gripping the object by the robot arm, the robot arm is moved to a position where the visual inspection sensor can image the object, and an image captured by the visual inspection sensor is stored in the storage unit. The process of detecting the current position when the position of the robot arm is controlled so as to match the target image, and the actual target position is obtained from the detected current position, the inspection reference position, and the target reference position. The position control of the robot arm is characterized in that Way. 4. An image of an object such as a work is previously photographed by a visual sensor attached to a robot arm and stored as a reference image, and the image of the object photographed by the visual sensor is the same as the reference image. A robot apparatus for performing visual feedback control for controlling the position and orientation of the robot arm so as to be in a state, in which a visual sensor for inspection installed in a fixed position and the object are The grasped robot arm is moved to a position where the inspection visual sensor can photograph the object, and the object is photographed by the inspection visual sensor at this position, and then the robot arm moves the object to the other side. When the instruction to move to the position for passing is executed, the image taken by the inspection visual sensor is stored as an inspection image and The storage means for storing the position of the robot arm and the position of the robot arm moved to pass the object to the other side as an inspection reference position and a target reference position, respectively, and by the visual feedback control during actual work. After grasping the object by the robot arm whose position and orientation are controlled, the robot arm is moved to a position where the inspection visual sensor can image the object, and a captured image by the inspection visual sensor is Position detecting means for detecting a current position when the position of the robot arm is controlled so as to match the inspection image stored in the storage means, the current position detected by the position detecting means, the inspection reference position and Calculation means for obtaining the actual target position from the target reference position and the actual target position obtained by this calculation means Robot apparatus characterized by comprising a control means for moving said robot arm.