JP2002071313A - Target marker, method for measuring location of object for using the same, and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a target marker, capable of easily distinguishing marks independently of the relation of angular location with a visual sensor and conducting distinguishing, without being affected by the reflection of the sunlight. SOLUTION: The target marker 1, arranged at an object and used for measuring the location of the object, is constituted of a marker base 2 given a first color (e.g., red) and a plurality of markers 3a-3d mounted onto the marker base 2. The markers 3a and 3b adjacent along the right side of the marker base 2 are given a second color (e.g., yellow), and the markers 3c and 3d adjacent along the left side of the marker base 2 are given a third color (e.g., green).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a target marker, a method for measuring a position of an object using the target marker, and a robot system.

[0002]

2. Description of the Related Art In general, when performing various assembling operations by a robot, an image of an object is acquired by a visual sensor for positioning a robot arm, and the image is binarized, and then contours (edges) are extracted. A method of obtaining a relative line image (including a posture) of the target object with respect to the robot arm is performed by obtaining a contour line image and performing matching with predetermined information of the target object.

[0003] In this method, there is a problem that the position measurement accuracy of the object largely depends on the shape of the object. For this reason, a method has been proposed in which a member called a target marker having a position detection mark formed on an object is installed, an image of the target marker is acquired by a visual sensor, and the relative position of the object is obtained by image processing. ing. As such an example, for example, Japanese Patent Application Laid-Open No. Hei 5-31521 describes a target marker in which a ring-shaped mark having the largest area and five circular marks having only one area different from the others are arranged on a base. .

In the known target marker, the types of a plurality of marks constituting the target marker are identified by the shape and the size. May not be recognized, and it may be difficult to identify the mark.

[0005] Further, in this known example, the color is not particularly mentioned, but since the mark is identified by the shape and size, it is considered that one of the base and the mark is white and the other is black. Can be However, with such a black-and-white target marker, for example, when applied to a robot that works in outer space, the marker is reflected by sunlight, and the marker itself is identified as a surrounding object,
It is expected that the accuracy in identifying each mark will be significantly reduced.

[0006]

As described above, in a target marker in which marks having different shapes and sizes are arranged on the base, it is difficult to identify the mark due to the angular positional relationship between the marker and the visual sensor. When applied to a robot used in outer space, there has been a problem that it is difficult to identify the target marker and other marks by reflection of sunlight.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a target marker which can easily identify a mark irrespective of the angular positional relationship with a visual sensor and can be identified without being affected by the reflection of sunlight. is there.

It is another object of the present invention to provide a position measuring method capable of easily and reliably measuring a relative position of an object using a target marker.

It is another object of the present invention to provide a robot system capable of securely holding an object and performing a predetermined operation based on a result of relative position measurement of the object by a target marker.

[0010]

In order to solve the above-mentioned problems, a target marker according to the present invention is provided with a base colored in a first color and disposed on the base, and at least one of the base markers is provided in a second color. The color is characterized in that at least one other has three or more marks respectively colored in a third color.

Further, in the position measuring method according to the present invention, the target marker is set on an object to be positioned, and the target marker is picked up by a color visual sensor composed of, for example, a stereo color CCD camera to obtain a color image. The position measurement of the object is performed based on the first to third components in the color image. In this position measurement, first, a coarse position measurement based on the first color component is performed, and then a fine position measurement including the posture of the object is performed based on the second and third color components. Is also good.

Further, the present invention relates to a robot system for performing a predetermined operation by gripping an object with the end of a robot arm, wherein the target marker is set on the object, and a color visual sensor installed near the end of the robot arm. By capturing a target marker to obtain a color image,
The acquired color image is input to the image processing unit, and the position of the target is measured based on the first to third color components in the color image. The drive control of the robot arm is performed based on this.

Here, the image processing section measures the coarse position of the object based on the first color component in the color image and the posture of the object based on the second and third color components in the color image. The robot arm drive control means performs coarse positioning of the gripper that grips the object of the robot arm based on the coarse position measurement result, and then performs fine positioning based on the fine position measurement result. The drive of the robot arm may be controlled so as to perform precise positioning of the robot arm.

More specifically, in the coarse position measurement, specifically, a color image obtained by imaging a target marker is compared with a reference color pattern registered in advance, and the reference color pattern of the color image is compared. Color pattern matching is performed so that the area includes only the same color as the reference color pattern, and a color histogram of the color image is created at the position matched by the color pattern matching, and the base color component of the marker in the image is created. The brightness of the marker is extracted from the color image, and an area in the same brightness range as the brightness of the color component is extracted from the color image, and the logical product of the image of the extracted area and the color image is calculated. The image is extracted and its center of gravity is calculated, and the target marker is calculated from the center of gravity and the center of the color vision sensor. Relative position, that is calculated as the three-dimensional position as a position of the object from the X-Y-Z coordinates.

On the other hand, in the fine position measurement, for example, in the same manner as in the coarse position measurement, color pattern matching is performed on a color image obtained by imaging a target marker, and a color histogram of the color image is obtained at the position matched by the color pattern matching. Is created, the brightness of the base color component of the marker in the image is determined, an area in the same brightness range as the brightness of the red component is extracted from the color image, and the logic of the extracted area and the entire color image is extracted. The product is calculated to extract an image of the first color region of the target marker, that is, the image of the base region, to identify the outline of the image of this region, to identify the base region, and to determine the color within the base region. An image of a mark portion colored in a second color and a third color by, for example, performing RGB-HLS conversion on the image. Extracted binarization to the centroid of the image, i.e., obtains the three-dimensional coordinates of the center of each of the marks to determine the three-dimensional center of the entire marked as the center of the target marker.

Then, a plane passing through each center of each mark is obtained, and a normal vector perpendicular to this plane and passing through the center of the target marker is obtained. Then, as two vectors perpendicular to this normal vector, A vector in the Y direction from the center of the two marks colored in the second color, and the center of the mark colored in the second color and the X direction from the center of the mark colored in the third color adjacent thereto. , The position of the object is measured as the relative distance and orientation of the target marker from the color vision sensor.

As described above, according to the present invention, on the base colored in the first color, at least one is colored in the second color and at least one is colored in the third color. By providing the above mark, it is easy to identify the mark irrespective of the angular positional relationship with the visual sensor, and a target marker that can be identified without being affected by the reflection of sunlight, such as a black and white target marker. It is possible to easily and reliably measure the relative position of the target object by using the target marker, and to perform the predetermined work by securely grasping the target object based on the result of the relative position measurement. A robot system can be constructed.

[0018]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a plan view showing a configuration of a target marker according to one embodiment of the present invention. The target marker 1 is placed on an object (not shown) and used for position measurement of the object, and has a rectangular (square in this example) plate-like marker base 2 and a marker base 2 on the surface of the marker base 2. Marks 3a, 3 arranged at four corners
b, 3c and 3d. Here, the entire surface of the marker base 2 or the surface on which the marks 3a, 3b, 3c, 3d are arranged is colored in a first color, for example, red (R).
The reason why the surface of the marker base 2 is colored red in this embodiment is that the color of the marker base 2 is used for rough relative position measurement and positioning (coarse position measurement / positioning) of an object on which the marker base 2 is installed. The reason is that it is desirable that the color has the highest visibility in outer space.

On the other hand, the marks 3a, 3b, 3c, 3d are
The marks 3a and 3b adjacent to the marker base 2 along the right side in the figure are colored in a second color, for example, yellow (Y). In the figure, the marks 3c and 3d adjacent to each other along the left side of the marker base 2 are colored in a third color, for example, green (G).

To illustrate the specific shape and dimensions of each part of the target marker 1, for example, the marker base 2 is a square having a side length of about 30 mm, the marks 3a, 3b, 3c, and 3c.
3d is a circle having a diameter of about 6 mm. Mark 3
The distance between the a, 3b, 3c, and 3d in the direction along the side of the marker base 2 is about 20 mm.

The marks 3a, 3b, 3c, 3d are formed, for example, in the form of a thin sheet and affixed on the marker base 2 so as not to protrude from the surface of the marker base 2, or directly on the surface of the marker base 2. It is desirable to be printed. Thus, the marks 3a, 3b, 3
By forming c and 3d so as not to protrude from the surface of the marker base 2, the marks 3a, 3b, 3c and 3d may hinder the movement of the arm when gripping an object with a robot arm described later. Disappears. The marks 3a, 3b, 3c, 3d are circular in the example shown in the figure, that is, isotropic, but are anisotropic, such as a square, a triangle, an ellipse, etc., that is, a shape having directionality. There may be.

FIG. 2 shows a state in which the target marker 1 shown in FIG. 1 is attached to an object 18, the relative position of the object 18 with respect to the robot arm 11 is measured, and the robot arm 1 based on the measurement.
1 shows the configuration of a robot system that performs positioning with respect to one target object 18 and grips a handle 19 of the target object 18 to perform predetermined operations such as transportation and assembly. This robot system can of course be used on the ground, but is particularly suitable for use in outer space.

In FIG. 2, the robot arm 11 has six joints 13a, 13b, 13c, 13d, 13e, 13f, an upper arm 14 and a lower arm 15 so as to have six degrees of freedom on the robot base 12. It has become. At the tip of the robot arm 11, an end effector 16 for gripping the handle 19 of the object 18 and performing a predetermined operation is provided.

The joints 13a, 13b, 13c, 13d,
13e and 13f will be further described.
a is the joint axis θ1 of the roll axis of the robot base 12 (first
Axis), the joint 13b is the joint axis θ2 of the pitch axis of the lower arm 15.
(Second axis), the joint 13c is the joint axis θ3 (third axis) of the pitch axis of the upper arm 14, and the joint 13d is the end effector 1.
A joint axis θ4 (fourth axis) of the pitch axis of No. 6 and a joint axis 13 of the yaw axis of the end effector 16 (fifth axis)
Axis) and the joint portion 13f have a joint axis θ6 (sixth axis) of the roll axis of the end effector 16, and are driven to rotate around respective rotation axes indicated by alternate long and short dash lines in FIG. 11 can change the position and posture of the tip. Also, each of the joints 13a, 13b,
Position sensors and angle sensors (not shown) are attached to 13c, 13d, 13e, and 13f.

A color vision sensor 17 is attached to the end effector 16. The color visual sensor 17 is constituted by, for example, a stereo color CCD camera, and acquires a color image including the target marker 1 attached to the object 18, that is, a color image (for example, an RGB image) of the target marker 1 and its peripheral portion. Then, a color image signal is output to the control device 21.

As shown in FIG. 3, the control unit 21 includes an image processing unit 22 for processing a color image signal input from the color visual sensor 17, and a drive control unit 23 for receiving an output signal from the image processing unit 22 as an input. , And the joints 13a, 13b, 13c, 13d,
The driving device 24 is configured to rotate the driving devices 13e and 13f.

Next, the operation of the robot system according to the present embodiment will be described. First, the target marker 1 on the object 18 is imaged by the color visual sensor 17 attached to the tip of the robot arm 11, and an RGB image including the target marker 1 is acquired. The image signal of the RGB image obtained by the color visual sensor 17 is input to the control device 21. The image signal input to the control device 21 is processed by the image processing unit 22, and the relative position of the target marker 1 with respect to the robot arm 11,
That is, the relative position of the object 18 is measured.

The drive unit 24 is controlled by the drive control unit 23 based on the measurement result of the relative position, so that
Joints 13a, 13b, 13c, 13d, 13e, 13
f is rotated and the robot arm 11 is positioned. Thereafter, the handle 19 of the target object 18 is gripped by the end effector 16, and a predetermined operation such as transportation and assembly of the target object 18 is performed.

The measurement of the relative position of the object 18 and the positioning of the robot arm 11 are performed in this embodiment in the order of the coarse position measurement / positioning mode and the fine position measurement / positioning mode. Hereinafter, a detailed procedure of each mode will be described.

(Coarse Position Measurement / Positioning Mode) In the coarse position measurement / positioning mode, for example, when the dimensions of each part of the target marker 1 are as described above (particularly when the length of one side of the marker base 2 is 30 mm), Used when the distance from the color vision sensor 17 to the target marker 1 is 200 mm or more. FIG. 4 shows a processing procedure in the coarse position measurement / positioning mode.

In this coarse position measurement / positioning mode, first, a color image signal, that is, an RGB image obtained by imaging the color visual sensor 17 composed of a stereo color CCD camera in the direction of the target marker 1 is fetched (step). S11).

Next, color pattern matching, that is, as shown in FIG. 5, the captured RGB image 40 is converted into a reference color pattern (a color pattern having the same color as the marker base 2 and a predetermined shape) 41 registered in advance. Then, the robot arm 11 is positioned so that the area of the reference color pattern 41 of the RGB image 40 includes only the same color as the reference color pattern 41 (step S12). In this example, the color of the reference color pattern 41 is set to red since the marker base 2 is red. The shape of the reference color pattern 41 is shown in FIG.
Although the shape is rectangular, it may be circular or another shape.

Next, the position matched by the color pattern matching in step S12, that is, the RGB image 40
In a state where the robot arm 11 is positioned so that the area of only the same color as the reference color pattern 41 is included in the range of the reference color pattern 41, as shown in FIG.
A color histogram of the B image 40 is created, and the brightness of the R (red) component, which is the color component of the marker base 2 in the RGB image 40, is checked (step S13). The color histogram of FIG. 6 has brightness (luminance) on the horizontal axis and frequency on the vertical axis. Brightness (brightness) depends on the color components,
In this example, the R (red) component has the maximum luminance.

Next, as shown in FIG. 7, an area in the same brightness range as the brightness of the red component obtained in step S13 is extracted from the RGB image 40, and the logical product of the extracted area image and the RGB image is extracted. To extract the image of the red area 42 of the target marker 1, that is, the image of the marker base 2 (step S14).

Next, the red area 42 of the target marker 1 extracted in step S14 (the part of the marker base 2)
Is calculated (step S15). For calculation of the center of gravity 43, various known image processing techniques can be used. For example, for the image of the red area 42, in the X direction, a weighted average of X coordinate values according to the number of pixels in the Y direction perpendicular to the X area And the average value is taken as the center of gravity in the X direction (X coordinate of the center of gravity 43).
The weighted average of the Y coordinate values according to the number of pixels in the direction may be obtained, and the average value may be used as the center of gravity in the Y direction (Y coordinate of the center of gravity 43).

Next, the center of gravity (center between the optical axis of the left camera and the optical axis of the right camera) of the stereo color CCD camera, which is the color visual sensor 17, and the center of gravity 43 of the marker base 2 calculated in step S15. From, the relative position of the target marker 1 with respect to the robot arm 11 is
It is calculated as a three-dimensional position consisting of Z coordinates (step S
16). Here, among the XYZ coordinates representing the relative position (three-dimensional position) of the target marker 1, X and Y are RGB.
It can be obtained as the XY coordinates of the center of gravity 43 of the marker base 2 in the image, and Z is the distance between the center of gravity of the stereo color CCD camera which is the color visual sensor 17 and the center of gravity 43 of the marker base 2 based on the principle of triangulation. Can be obtained as

By the above steps S11 to S16, the approximate relative position of the target marker 1 with respect to the robot arm 11 is determined. That is, the rough position measurement of the relative position of the target object 18 with respect to the robot arm 11 has been performed.

Then, the current position (three-dimensional position) of the robot arm 11 and the command position generated in advance with reference to the relative position (three-dimensional position) of the target marker 1 are used to determine the tip of the robot arm 11 (end effector). 1
Positioning (coarse positioning) is performed only for step 6) (step S17).

Specifically, a drive control unit in the control device 21
23, each joint 13a of the robot arm 11
Detects positions of 13b, 13c, 13d, 13e, 13f
Base seat calculated from the output of a position sensor (not shown)
Target system (coordinate system based on robot base 12) (Σba
se) the position of the tip of the robot arm 11^baseT_tipWhen,
Mounting of the color vision sensor 17 on the robot arm 11
Arm coordinate system (robot arm
With reference to the position of the end effector 16 which is the tip of 11
Position of the color visual sensor 17 in the coordinate system (tip)
Place^tipT_cameraBy using the tip of the robot arm 11
Arm tip position command indicating target position^baseT _{tip demand}But
It is calculated by the following equation.^baseT_{tip demand}=^tipT
_camera・^cameraT_marker・^markerT_{tip demand}further,
Arm tip position command calculated in this way^baseT
_{tip demand}Of the tip of the robot arm 11 based on
An arm tip speed command indicating the speed is generated.

In the coarse position measurement / positioning mode, as described above, only the relative position of the target marker 1 in the visual sensor coordinate system (coordinate system based on the position of the color visual sensor 17) (よ う camera) is measured. Marker 1
Is not measured. Therefore, the color vision sensor 17
The position of the target marker 1 in the visual sensor coordinate system (eracamera) measured by using the ^camera T _marker and the previously specified marker coordinate system (the coordinate system based on the position of the target marker 1) (Σmarker) Arm tip position command
For the posture term of the ^marker T _{tip demand} , an arm tip speed command is generated on the assumption that the target marker 1 faces the color visual sensor 17.

That is, the visual sensor coordinate system (Σcamera)
Of the target marker 1 in the marker coordinate system (系 ma
rker) The arm tip position command ^marker T _{tip demand} is
The calculation is performed as if the respective coordinate systems (座標 camera) and (Σmarker) face each other.

The robot arm 11 is roughly positioned by inputting the thus generated arm tip speed command from the drive control unit 22 in the control device 21 to the drive device 23.

(Fine Position Measurement / Positioning Mode) In the fine position measurement / positioning mode, for example, when the dimensions of each part of the target marker 1 are the above-described example (particularly, when the length of one side of the marker base 2 is 30 mm), It is used when the distance from the color vision sensor 17 to the target marker 1 approaches within 200 mm. Therefore, if the distance from the color visual sensor 17 to the target marker 1 is within 200 mm in the initial state of the robot system 11,
The coarse position measurement / positioning mode described above is omitted, and only the fine position measurement / positioning mode is used.

FIG. 8 shows a processing procedure in the precise position measurement / positioning mode. In the fine position measurement / positioning mode, the processing in steps S21 to S23 is performed in step S1 in the coarse position measurement / positioning mode shown in FIG.
This is the same as the processing of 1 to S13.

That is, the RGB image acquired by the color visual sensor 17 is fetched (step S21),
This RGB image is subjected to color pattern matching (step S22), and a color histogram is created at this matching position to determine the brightness of the red component, which is the color of the marker base 2 (step S23). An area in the same brightness range as the brightness of the red component determined in step S23 is extracted from the RGB image 40, and the logical product of the extracted area and the RGB image 40 is calculated, and the red area 42 of the target marker 1 ( The image of the marker base 2 (area of the marker base 2) is extracted, and the outline of the image of the red area 42 is determined to identify the red area 42, that is, the area of the marker base 2 (step S24).

In the fine position measurement / positioning mode, the image inside the area of the marker base 2 identified in step S24 is subjected to RGB-HLS (hue) conversion to obtain an image.
As shown in FIG. 9, yellow areas, that is, marks 3a and 3b
Is extracted and binarized (step S
25).

Next, the center of gravity of the image of the binarized yellow area obtained in step S25, that is, the marks 3a, 3
The center of each of b is obtained (step S26).

Next, as in step S25, the image inside the area of the marker base 2 identified in step S24 is subjected to RGB-HLS (hue) conversion to obtain the image shown in FIG.
As shown in (2), the image of the green area, that is, the image of the mark 3c, 3d is extracted and binarized (step S2).
7).

Next, similarly to step S26, the center of gravity of the binarized green area image obtained in step S27, that is, the center of each of the marks 3c and 3d is obtained (step S28).

The processes in steps S25 to S28 are performed on the RGB images acquired by the left and right cameras of the stereo color CCD camera constituting the color vision sensor 17. Thereby, the four marks 3a, 3b, 3c,
The center of each of 3d is obtained as three-dimensional coordinates. Then, next, the marks 3a, 3b, 3c, 3d are obtained from the three-dimensional coordinates of the center of each of the four marks 3a, 3b, 3c, 3d.
The three-dimensional center of the whole is determined as the center of the target marker 1 (step S29).

Next, the four marks 3a, 3b, 3c, 3
A plane passing through each center of d is obtained by, for example, the least square method,
A normal vector perpendicular to this plane and passing through the center of the target marker 1 obtained in step S29 is obtained (step S30).

Next, 2 perpendicular to the normal vector
Two vectors are obtained (step S31). That is,
The two yellow marks 3a obtained in step S26,
From the center of 3b, a vector in the Y direction (vertical direction) is obtained, and the center of one yellow mark (for example, mark 3a) and a green mark ( For example, a vector in the X direction (horizontal direction) is obtained from the center of the mark 3d).

By the above steps S21 to S31, the relative position (relative distance and attitude) of the target marker 1 from the color visual sensor 17 has been obtained. Then, next, the current position (three-dimensional position) of the robot arm 11
And the robot arm 1 based on the relative distance and posture of the target marker 1 obtained in steps S21 to S33.
Robot base 1 of 1 tip (end effector 16)
Then, a command position is generated based on 2 and the entire robot arm 11 is positioned based on the command position, that is, precise positioning is performed (step S32).

Specifically, a drive control unit in the control device 21
23, each joint 13a of the robot arm 11
Detect angles of 13b, 13c, 13d, 13e, 13f
Base seat calculated from the output of an angle sensor (not shown)
Target system (coordinate system based on robot base 12) (Σba
arm tip position at se)^baseT_tipAnd the robot arm
11 depends on the mounting state of the color vision sensor 17
Completely coordinate the tip of the arm (the position of the end effector 16
Color vision sensor 1 with reference coordinate system (Σtip)
7 position^tipT_cameraAnd using the color vision sensor 17
Sensor coordinate system (color visual sensor 17
Target marker in the reference coordinate system (基準 camera)
1 position^cameraT_markerAnd a previously specified marker coordinate system
(Coordinate system based on the position of the target marker 1) (Σ
marker) arm tip position command ^markerT_{tip demand}To
Using the arm tip position command^baseT_{tip demand}Becomes
It is calculated from:^base T_{tip demand}=^baseT_tip・^tipT_camera・^cameraT
_marker・^markerT_{ti p demand} Furthermore, this arm tip position command^baseT_{tip demand}Based on
Then, an arm tip speed command is created. Generated in this way
The set arm tip speed command is used for drive control in the controller 21.
By being input to the driving device 23 from the unit 22, the robot
The precise positioning of the entire cut arm 11 is performed.

As described above, according to the present embodiment, the marker base 2 in which the target marker 1 is colored in the first color (for example, red), and the plurality of markers 3a, 3b, 3c, 3d, and adjacent marks 3a, 3 along the right side of the marker base 2.
b is colored in a second color (for example, yellow), and the marks 3c and 3d adjacent along the left side of the marker base 2 are colored in the third color.
(For example, green).

Then, a color image (for example, an RGB image) of the target marker 1 is obtained by the color visual sensor 17, and the color of the marker base 2 and the marks 3a, 3b, 3 are obtained.
c, 3d and each mark 3a, 3b, 3c, 3
The rough position and the fine position of the object 18 are measured using the color difference d.

Accordingly, for example, when the target marker 1 is obliquely positioned relative to the visual sensor 17, the marker base 2 and the marks 3a, 3b, 3c, 3d may be deformed, for example, in a circular shape, and may look like an ellipse. Even if the magnitude relationship looks different, the marker base 2 and the marks 3a, 3b, 3c, 3
d can be identified.

Further, the marker base 2 and the marks 3a, 3
Since b, 3c and 3d are each colored in a unique color (chromatic color), even when applied to, for example, a robot system mounted on a satellite in outer space, it is affected by the reflection of sunlight. And these can be reliably identified.

Therefore, the position of the object 18 based on the color image of the target marker 1 acquired by the color visual sensor 17 can be measured with high reliability.
The gripping operation of the target object by the robot arm 11 of the robot system can be reliably performed.

The present invention is not limited to the above embodiment, but can be implemented with various modifications. For example, various modifications as shown in FIG. 11 are possible for the target marker in addition to the configuration shown in FIG.

FIG. 11A shows an example in which one of the two green marks 3c and 3d in the target marker 1 shown in FIG. 1 is omitted, and FIG. 11B shows the target marker shown in FIG. 1, two yellow marks 3a,
This is an example in which one of 3b is omitted. Thus, the number of marks in the target marker may be three, at least one of which is the second color and the other at least one of which is the third color.

FIG. 11 (c) is an example in which the target marker 1 of FIG. 11 (a) is modified and three marks 3a, 3b, 3c are arranged at three vertices of an equilateral triangle, respectively. In this case, when two vectors perpendicular to the normal vector are obtained in step S31 in the above-described fine position measurement / positioning mode, for example, for the vector in the Y direction (vertical direction), the two vectors obtained in step S26 are obtained. The vector between the centers of the yellow marks 3a and 3b may be obtained by converting it into a vector in the Y direction (vertical direction).

FIG. 11D shows an example in which the shape of the marker base 2 is circular. In this case, for example, in the coarse position measurement / positioning mode and the fine position measurement / positioning mode, the center of gravity 43 of the red area 42 (the area of the marker base 2) of the target marker 1 is calculated in step S15.
Can be realized by determining the center of gravity of the image of the red area 42 by known image processing. Further, the shape of the marker base 2 may be a rectangle, a diamond, an ellipse, or the like.

Further, in the target marker described above, the first color which is the marker base color is red, and the second and third colors which are the mark colors are yellow and green. Needless to say, can be changed arbitrarily.

[0065]

As described above, according to the present invention, a target marker which can easily identify a mark regardless of the angular positional relationship with a visual sensor and can be identified without being affected by the reflection of sunlight. Robot that can easily and reliably measure the relative position of the target object using the target marker, and further performs a predetermined operation by securely gripping the target object based on the result of the relative position measurement. A system can be provided.

[Brief description of the drawings]

FIG. 1 is a plan view showing a configuration of a target marker according to an embodiment of the present invention.

FIG. 2 is a diagram showing a schematic configuration of a robot system according to the embodiment;

FIG. 3 is a block diagram showing a configuration of a control device shown in FIG. 2;

FIG. 4 is a flowchart showing a processing procedure in a coarse position measurement / positioning mode in the embodiment.

FIG. 5 is a view for explaining a color pattern matching process in step S12 in FIG. 4;

FIG. 6 is a diagram showing an example of a color histogram created in step S13 in FIG.

FIG. 7 is a view for explaining processing in steps S14 to S15 in FIG. 4;

FIG. 8 is a flowchart showing a processing procedure in a precise position measurement / positioning mode in the embodiment.

FIG. 9 is a view for explaining the processing of steps S25 to S26 in FIG. 8;

FIG. 10 is a view for explaining processing in steps S27 to S28 in FIG. 8;

FIG. 11 is a plan view showing a configuration of a target marker according to another embodiment of the present invention.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Target marker 2 ... Marker base (red) 3a, 3b ... Mark (yellow) 3c, 3d ... Mark (green) 11 ... Robot arm 12 ... Robot base 13a to 13f ... Joint part 14 ... Upper arm 15 ... Lower arm 16 ... End effector (arm tip) 17 ... Color visual sensor 18 ... Object 19 ... Handle 21 ... Control device 22 ... Image processing unit 23 ... Drive control unit 24 ... Drive device 40 ... RGB image 41 ... Reference color pattern 42 ... Red area ( (Marker base portion) 43 ... center of gravity of red area

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 7/60 150 G06T 7/60 150B (72) Inventor Fumio Ozaki Komukai Toshiba-cho, Kochi-ku, Kawasaki-shi, Kanagawa No. 1 Toshiba R & D Center (72) Inventor Yasunori Takase 33 Shinisogo-cho, Isogo-ku, Yokohama-shi, Kanagawa Prefecture F-term (reference) 2F065 AA04 AA14 AA37 AA56 BB02 BB28 DD04 DD09 DD11 FF01 FF05 FF09 FF61 JJ03 JJ05 JJ08 JJ26 NN06 QQ25 QQ36 QQ38 QQ43 RR02 RR08 UU04 UU05 3F059 AA20 BA02 BA10 CA05 DA02 DA09 DB04 DB09 DC08 DD01 DE03 FA03 FA10 FB12 FB22 CA13 FA03GA02A

Claims (5)

[Claims] 1. A target marker installed on an object and used for position measurement of the object, a base colored in a first color, and disposed on the base, at least one of which is provided on the base. A target marker having a second color and three or more marks each of which is at least one other colored in a third color. 2. A base colored on a first color and three or more colored bases disposed on the base, at least one of which is colored with a second color and at least one of which is colored with a third color. Placing a target marker composed of a mark on a positioning target, capturing the target marker to obtain a color image, and setting the target based on the first to third color components in the color image. A position measuring step of measuring a position of an object. 3. The method according to claim 1, wherein the position measuring step includes a step of measuring a coarse position of the object based on the first color component in the color image, and the first to third steps in the color image. Performing a precise position measurement including a posture of the object based on a color component. 4. A robot system for performing a predetermined operation by gripping an object with the tip of a robot arm, wherein the base is provided on the object and is arranged on the base, the base being colored in a first color; One is a target marker composed of three or more marks colored in a second color and at least one is colored in a third color, and an image of the target marker is provided near the tip of the robot arm. A color visual sensor for obtaining a color image by performing the measurement, and an image processing unit for performing position measurement of the object based on the first to third color components in the color image obtained by the color visual sensor. And a drive control means for driving and controlling the robot arm based on a result of the position measurement of the object by the image processing unit. . 5. The image processing section, after performing a coarse position measurement of the object based on the component of the first color in the color image, sets the first to third positions in the color image. The coarse position measurement including the posture of the object is performed based on the color component, and the drive control unit performs coarse positioning of a gripper that grips the object of the robot arm based on the coarse position measurement result. After performing the operation, the robot system controls the driving of the robot arm so as to perform the precise positioning of the robot arm based on the result of the accurate position measurement.