JPH05138582A - Active camera searching device for manipulating object by using robot - Google Patents

Abstract

PURPOSE:To provide an active camera searching device for manipulating an object with the use of a robot which actively find out the object so as to simply, stably and precisely carry out visual feed-back control. CONSTITUTION:A robot 1 and an object 2 are photographed by a camera 5, and thus photographed outputs are image-processed in an image processing part 9 so as to obtain a relative positional relationship between the robot 1 and the object 2. A robot control part 4 causes the robot 1 to approach the object 2. When the robot 1 comes onto the object 2, the camera 2 is moved under the control of a camera control part 7 so as to again obtain a relative positional relationship between the robot 1 and the object 2, then causing the robot 1 to approach the object 2. If the robot 1 fails to recognizes the object due to the presence of an obstruct, the camera 5 is moved upward, and then the relative positional relationship between the robot 1 and the object 2 is obtained.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an active camera search device for object manipulation by a robot, and more particularly, to make visual feedback control of object manipulation simpler, more stable, and more accurate by the active camera search, thereby making it possible to perform complicated object manipulation. The present invention relates to an active camera search device for object manipulation by a robot that can be adapted to the arrangement.

[0002]

2. Description of the Related Art Recently, in order to manipulate a target object, the target object is imaged by a camera, the image is processed, and feedback control is performed to move the camera. , I am moving the robot. However, when the target object is imaged by one fixed camera, the three-dimensional position of the target object cannot be specified. That is, even if the relative positions of the target object and the robot are two-dimensionally matched, it is not possible to determine whether the relative positions of the target object and the robot are matched in the three-dimensional direction, that is, the depth direction.

A method of measuring the three-dimensional position of a robot and a target object in a stereo system using two cameras has been considered, but if the baseline between the two cameras is lengthened, two screens of the object are displayed. If it becomes difficult to associate the images with each other, and if they are shortened, the accuracy of the three-dimensional position obtained by triangulation may be low. In addition, the fixed relationship between the target object and the camera causes problems such as the object being hidden, being out of the field of view, and having insufficient image resolution.

Therefore, a main object of the present invention is to operate an object by a robot such that a camera is actively searched for an object to perform visual feedback control more simply, stably, accurately and efficiently. It is to provide an active camera search device for.

[0005]

SUMMARY OF THE INVENTION The present invention is an active camera search device for object manipulation by a robot, wherein a robot manipulates a target object while imaging the target object with a camera, and processes the image output of the camera. The image processing means, the camera control means for controlling the camera, the robot control means for controlling the robot, and the relative positional relationship between the robot and the target object in accordance with the image signal processed by the image processing means. Control means for controlling the robot so that the robot approaches the target object, the camera control means controls to change the imaging position of the camera, and the robot control means controls the robot so that the robot approaches the target object again. Composed.

[0006]

The active camera search device for object manipulation by the robot according to the present invention processes the image output of the camera to obtain the two-dimensional relative positional relationship between the robot and the target object, and the robot approaches the target object. By controlling the robot in such a manner that the imaging position of the camera is changed so that the robot approaches the target object again, the three-dimensional positional relationship between the robot and the target object can be simplified, stable and accurate with one camera. Can be measured.

[0007]

1 is a schematic block diagram of an embodiment of the present invention. Referring to FIG. 1, the robot 1 performs a three-dimensional operation in order to grab the target object 2 and place it at a designated position. For this reason, the robot control unit 4, when given a command from the computer 3, performs control for bringing the robot 1 closer to the target object 3.

On the other hand, the camera 5 is attached to the camera arm 6 and images the robot 1 and the target object 2. Camera 5
The continuous image 8 picked up in (1) is given to the image processing unit 9 and subjected to image processing. The image processing unit 9 extracts the features of the target object captured by the camera 5 and processes the continuous images to track the movements of the robot 1 and the target object 2. The output of the image processing unit 9 is given to the computer 3, and the camera 5
Controls the camera control unit 7 so as to track the movement of the robot 1.

FIG. 2 is a diagram showing camera movement in the camera center coordinate system. Referring to FIG. 2, the camera 5 is the target object 2
In order to change the image resolution when the image of the target image is captured, the camera 5 is moved to the target object 2 in a state where the movement 10 along the optical axis is in focus and the position of the target object is adjusted.
Movement 11 that rotates in the vertical direction around the target object 2
It is possible to make a movement 12 along a horizontal arc around the gazing point selected to determine the relative position of the robot 1.

FIG. 3 is a diagram showing an example of an image picked up by a camera. In the image 13 taken by the camera 5, the robot 1 and the target object 2 are shown. The camera 5 moves along the optical axis and is enlarged when approaching the target object 2, and is reduced when moving away from it. Then, when the screen size of the target object becomes a constant value, the movement in the optical axis direction is stopped. By rotating the camera 5 around the vertical axis or the horizontal axis to change the observation direction, the center of the image capturing range 13 and the center of the image processing range 14 can be matched.

FIG. 4 is a diagram showing the relationship between the camera center coordinate system and the robot center coordinate system, FIG. 5 is a diagram for explaining relative movement between the camera and the robot, and FIG. 6 is a diagram showing the camera. FIG. 9 is a diagram for explaining the operation of the robot grasping the target object by the guidance of FIG. 7, and FIG. 7 is a flowchart for explaining the operation of the embodiment of the present invention.

Next, the operation of the embodiment of the present invention will be described with reference to FIGS. In step (abbreviated as SP in the figure) SP1, the camera 5 images the robot 1 and the target object 2. The image captured by the camera 5 at this time is as shown in FIG. The camera 5 is moved so that the robot 1 and the target object 2 are at the center of the image. The image at that time is shown in FIG.
As shown in. Next, the robot 1 is moved along three primary independent vectors T1, T2, T3 in space as shown in FIG. 4, and the movement state is imaged by the camera 5,
The relationship between the camera center coordinate system and the movement center coordinate system of the robot 1 is obtained. That is, the image processing unit 9 calculates the vector on the screen by multiplying the vectors T1, T2, T3 in the movement center coordinate system of the robot 1 by the rotation matrix. Then, the camera 5 is moved so that the tip position of the robot 1 becomes the center position on the screen. As a result, the image becomes as shown in FIG. In step SP4, the tip position of the robot 1 is brought close to the target object 2. At this time, the robot 1
Either the camera 1 or the robot 5 is moved as shown in FIG. 5 so that the tip of the robot is on the same vertical line as the target position point of the target object 2. When the tip of the robot 1 and the target object 2 are on the same vertical line, the image becomes as shown in FIG. 6 (d).

At step SP5, it is determined whether or not the robot 1 has arrived on the target object 5. When it is determined that the difference between the robot 1 and the target object 2 has disappeared, in step SP6, the camera 5 is moved in the horizontal direction while keeping the distance from the target object 2 to the camera 5 constant.
Then, as shown in FIG. 6E, when it is determined that there is an obstacle between the robot 1 and the camera 5 and the image of the robot 1 cannot be recognized, the height of the camera 5 is determined in step SP8. Change. That is, the camera 5 is moved upward while keeping the distance from the target object 2 to the camera 5 constant. Then, as shown in FIG. 6 (f),
The image of the robot 1 can be recognized from the direction above the obstacle. In step SP9, the robot 1 is brought close to the target object, and in step SP10, the robot 1
It is determined whether or not the vertical axis of 1 and the vertical axis of the target object 2 match, and if they match, the robot 1 has reached the position of the target object 2. Then, the robot 1 picks up the target object 2 and moves it to another position.

[0014]

As described above, according to the present invention, the image of the target object and the robot are picked up by the camera, the image output is processed to obtain the relative positional relationship between them, and the robot is made to approach the target object. By controlling the robot to change the image pickup position of the camera so that the two are coincident with each other and the robot is controlled to approach the target object, it is possible to perform feedback control with one camera simply, stably and accurately. Therefore, it is possible to solve problems such as occlusion that cannot be solved by a single fixed camera as in the related art.

[Brief description of drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a diagram showing camera movement in a camera center coordinate system.

FIG. 3 is a diagram illustrating an example of an image captured by a camera.

FIG. 4 is a diagram showing a relationship between a camera center coordinate system and a robot center coordinate system.

FIG. 5 is a diagram for explaining relative movement between a camera and a robot.

FIG. 6 is a diagram for explaining an operation in which a robot grabs a target object by guiding a camera.

FIG. 7 is a flowchart for explaining the operation of the embodiment of the present invention.

[Explanation of symbols]

 1 Robot 2 Target Object 3 Computer 4 Robot Control Section 5 Camera 6 Camera Arm 7 Camera Control Section 9 Image Processing Section

Claims (1)

[Claims] 1. An active camera search device for object manipulation by a robot, wherein a robot manipulates the object while imaging the target object with the camera, the image processing means processing image output of the camera, A camera control means for controlling the robot, a robot control means for controlling the robot, and a relative positional relationship between the robot and the target object in accordance with an image signal processed by the image processing means, and the robot control means The robot is controlled to approach the target object, the camera control unit is controlled to change the imaging position of the camera, and the robot control unit is again controlled to approach the robot to the target object. Active Camera Search for Object Manipulation by Robot with Control Mechanism Search equipment.