TWI656421B - Control method of self-propelled equipment - Google Patents

Abstract

A control method of a self-propelled device is suitable for a self-propelled vehicle, a robot arm provided on the self-propelled vehicle, an image capture unit provided on the robot arm, and a workpiece, and includes the following steps: borrowing Moving from the self-propelled vehicle to an actual position, the actual position has an error amount relative to a predetermined position; an image is captured in an actual direction by the image capturing unit; the image is processed by the self-propelled vehicle Image recognition to obtain the relative positional relationship between the workpiece and the image capturing unit; the self-propelled vehicle obtains the error amount according to the relative positional relationship to control the robotic arm to actuate the workpiece. Therefore, a technology using optical tracking is implemented to improve the positioning accuracy of the self-propelled device.

Description

Control method of self-propelled equipment

The invention relates to a method for controlling an autonomous device, and in particular to a method for controlling the positioning accuracy of an autonomous device using optical tracking technology.

Automated Guided Vehicle (AGV) is a conventional device widely used in storage systems or factory manufacturing. The conventional self-propelled vehicle can move on the track, or it can move between different positions without the assistance of the track, so that the robotic arm provided on the self-propelled vehicle can perform the clamping at the correct position. Take or move a workpiece.

For example, when the self-propelled vehicle is to be moved from a first position to a second position, the following two methods are usually used to achieve the positioning of the self-propelled vehicle. In the first method, at least one triangular plate is provided near the second position, and at least one laser light source is provided on the self-propelled vehicle. When the laser light generated by the at least one laser light source is on the corresponding at least one triangle plate The upward reflection enables the sensor on the self-propelled vehicle to detect that it has moved to the second position. The second method is to set the coordinates of the second position directly in the control unit of the self-propelled vehicle, so that the self-propelled vehicle moves directly to the second position.

Because each time the self-propelled vehicle moves, it will produce non-fixed error values. For example, it moves more or less by a few centimeters in a certain horizontal direction, or the position of the self-propelled vehicle deviates although it is correct. This makes the robotic arm unable to perform gripping correctly. Of the two methods mentioned above, the second method does not compensate for the positioning of the self-propelled vehicle at all, while the first method has the position compensation by using the technology of laser reflection, but it has its practical implementation. Relative restrictions and inconveniences.

Therefore, an object of the present invention is to provide a control method that uses optical tracking technology to improve positioning accuracy.

Therefore, the control method of the self-propelled equipment of the present invention is applicable to a self-propelled vehicle, a robot arm provided on the self-propelled vehicle, an image capture unit provided on the robot arm, and a workpiece at a target position The control method includes steps (a) to (d).

In step (a), the self-propelled vehicle receives a driving signal and moves to an actual position. The actual position has an error amount relative to a predetermined position.

In step (b), an image is captured in an actual direction by the image capture unit. When the self-propelled vehicle is at the predetermined position, the actual direction is pointed by the image capture unit to the workpiece.

In step (c), the image is identified by the self-propelled vehicle to obtain a relative position relationship between the workpiece and the image capturing unit.

In step (d), the self-propelled vehicle obtains the error amount according to the relative position relationship to control the robot arm to act.

In some implementation aspects, the control method is also applicable to a mark adjacent to the workpiece, wherein in step (b), the actual direction is directed by the image capturing unit to the mark. In step (c), the self-propelled vehicle recognizes the image of the mark in the image to obtain the relative position relationship.

In some implementation aspects, in step (c), when the self-propelled vehicle is in the image and the mark cannot be recognized, the self-propelled vehicle controls the movement of the robotic arm until the self-propelled vehicle responds to the In the analysis performed on the images captured by the image capturing unit, the mark is recognized.

In some implementation forms, in step (c), the self-propelled vehicle controls the movement of the robot arm, so that the image capturing unit captures the images in turn for a plurality of predetermined areas. Define a virtual plane area, the normal direction of the virtual plane area is parallel to the actual direction, the virtual plane area contains the predetermined areas, and one of the predetermined areas of the virtual plane area will pass the location of the mark .

In some embodiments, in step (c), the size of the virtual plane area is related to a maximum moving error amount, a maximum rotating error amount of the self-propelled vehicle, and a ratio between the target position and the predetermined position. Distance.

In some implementation forms, in step (c), the number of the predetermined areas included in the virtual plane area is related to the size of the virtual plane area and the size of each image in the virtual plane area. The ratio of sizes.

In other embodiments, in step (c), the mark may be a Chessboard map, a QR code, a bar code, and an Aruco code. Either.

The effect of the present invention is: by using the image captured by the image capture unit for image recognition, the relative position relationship between the workpiece and the image capture unit is obtained, and then the predetermined position and the actual position can be obtained The amount of error. In addition, when a single image cannot be identified successfully, by controlling the movement of the robot arm, another updated image is obtained and image recognition is performed again until the identification is successful. Therefore, a technology using optical tracking can be implemented to improve the positioning accuracy of the self-propelled device.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are represented by the same numbers.

Referring to FIG. 1 and FIG. 2, FIG. 2 is a schematic plan view illustrating the positional relationship between a self-propelled device and a workpiece 7 by way of example. The control method of the self-propelled device of the present invention is applicable to the self-propelled device. The self-propelled device includes an Automated Guided Vehicle (AGV) 1, a robot arm provided on the self-propelled vehicle 1, and a setting. An image capturing unit 3 on the robot arm 2. In this embodiment, the image capturing unit 3 is a camera module for capturing a still image. In addition, the self-propelled vehicle 1 includes a processing unit (not shown) for receiving and sending signals and processing operations. To control the movement of the robot arm 2 and itself, but not limited to this. The control method includes steps S1 to S4.

In step S1, the self-propelled vehicle 1 receives a driving signal and moves to an actual position. The actual position has an error amount relative to a predetermined position. Referring again to FIG. 3, FIG. 3 is a coordinate diagram in a plan view, and illustrates a relative relationship between the predetermined position O1 and the actual position O2 by way of example. For example, the self-propelled vehicle 1 is to be moved from an initial position numbered 1 to 3 on the left side of FIG. 2 to the predetermined position numbered 1 to 3 on the right side, so that the robotic arm 2 can be positioned on a platform 8 The workpiece 7 performs gripping or other actions. However, during actual movement, the self-propelled vehicle 1 cannot just move to the predetermined position O1 every time, but moves to the actual position O2 of FIG. 3.

In addition, it should be added that the driving signal is, for example, a known technique such as a preset position coordinate or a start signal from another device, etc., so that the self-propelled vehicle 1 is driven to learn to move to the predetermined position. Influence or limit the technical means of the present invention.

Referring to FIG. 1, FIG. 2, and FIG. 3, the X1 direction and the Y1 direction of FIG. 3 are the coordinate systems with the self-propelled vehicle 1 as the reference point when the self-propelled vehicle 1 is at the predetermined position O1, and the X2 and Y2 directions It is the coordinate system that uses the self-propelled vehicle 1 as a reference point when the self-propelled vehicle 1 is at the actual position O2. In step S2, an image is captured by the image capturing unit 3 in an actual direction. In more detail, for example, when the self-propelled vehicle 1 is in the initial position, the image capturing unit 3 is oriented toward the actual direction. It is assumed that the self-propelled vehicle 1 is oriented toward an X1 direction as shown in FIG. 2 and FIG. 3. If the error amount is equal to zero, that is, when the self-propelled vehicle 1 moves to the predetermined position O1, the direction in which the image capturing unit 3 points to the workpiece 7 is the actual direction. In other words, during the movement of the self-propelled vehicle 1 from the initial position to the actual position O2, the image capturing unit 3 does not change the relative position with respect to the robot arm 2 or with respect to the self-propelled vehicle 1. relationship.

In step S3, the image is recognized by the self-propelled vehicle 1 to obtain a relative position relationship between the workpiece 7 and the image capturing unit 3. In this embodiment, a mark 6 adjacent to the workpiece 7 is also provided on the platform 8. The self-propelled vehicle 1 is used to identify the mark 6 in the image to obtain the relative position relationship. The mark 6 may be any one of a Chessboard map, a QR code, a bar code, or an Aruco code, so that the self-propelled vehicle 1 can be used for image recognition, but Not in this limit.

For the convenience of description, in the description of step S3, because the relative positions between the image capture unit 3, the robot arm 2, and the self-propelled vehicle 1 can be obtained in advance, therefore, it is assumed that the image capture The predetermined position O1 and the actual position O2 of the taking unit 3, the robot arm 2, and the self-propelled vehicle 1 are represented by O1 and O2 respectively in the coordinate diagram of FIG. 3, and will not affect the actual execution Feasibility of the invention.

Because the self-propelled vehicle 1 generates a translation error amount and a rotation error amount after moving. For example, dx and dy in FIG. 3 are the movement error amounts in the X1 direction and the Y1 direction, respectively, and dθ is the rotation error amount. Therefore, when the self-propelled vehicle 1 captures the image in the actual direction at the actual position O2, the image may not include the mark 6. At this time, that is, when the self-propelled vehicle 1 cannot recognize the mark 6 in the image, the self-propelled vehicle 1 controls the robotic arm 2 to move, that is, the image capturing unit 3 is directed to different directions or Position to capture another image. The self-propelled vehicle 1 performs image recognition on the other image. If the mark 6 is still not recognized, the robot arm 2 is controlled to move again to obtain the next image. In this way, the cyclic operations of the robotic arm 2 movement, acquisition of new images, and image recognition are sequentially controlled until the self-propelled vehicle 1 analyzes the images captured by the image capture unit 3 and recognizes that Up to this mark 6.

When the self-propelled vehicle 1 controls the movement of the robotic arm 2 so that the image capturing unit 3 captures a new image, the control method is to let the image capturing unit 3 make multiple reservations for a virtual plane area. The areas take turns capturing these images. The virtual plane will pass through the position of the marker 6. Taking the example of FIG. 3 as an example, suppose T represents the position of the marker 6. When the self-propelled vehicle 1 moves to the actual position O2, the image capturing unit 3 is located. The actual direction to which the position of O2 is (i.e., O2) is the Y2 direction. Therefore, the virtual plane area will pass the position T, and its normal direction is parallel to the Y2 direction.

The size of the virtual plane area can be obtained by a maximum movement error amount and a maximum rotation error amount in the specifications of the self-propelled vehicle 1 and the distance between the position T of the mark 6 and the predetermined position O1. For example, the larger the maximum movement error amount, the maximum rotation error amount, or the distance between the two positions, the larger the size of the virtual plane area.

More specifically, define P _Marker , P _Cam , P _Tool , P _Base , and P _AGV , a total of six coordinate positions, each coordinate position is based on the same reference point, and all have six dimensions, respectively It is the position in three mutually perpendicular directions of X, Y, and Z, and the rotation amounts corresponding to the three directions of X, Y, and Z, respectively. P _Marker is the coordinate position of the marker 6, P _Cam is the coordinate position of the image capture unit 3. When the image capture unit 3 is a clamping part set up in the last dimension of the robot arm 2, the P _Tool is The coordinate position of the clamping part, P _Base is the coordinate position of a bottom fulcrum of the robot arm 2, and P _AGV is the coordinate position of the self-propelled vehicle 1 (for example, the center of mass).

Defining T _M2C , T _C2T , T _T2B , T _B2A , there are four transformation matrices. The positions of these coordinates have the following relationship. Among them, T _B2A and T _C2T are both fixed relative positional relationship with respect to the self-propelled vehicle 1 and fixed relative positional relationship between the image capturing unit 3 and the clamping portion. A constant that can be obtained in advance. Furthermore, since the movement of the robot arm 2 is controlled by the self-propelled vehicle 1, the relative position relationship of the robot arm 2 with respect to the fulcrum of the aircraft is also known, that is, T _T2B is also known. Furthermore, T _M2C can be obtained by PnP (Perspective-n-Point) algorithm when the self-propelled vehicle 1 is used for image recognition, so it is also known.

From the above transformation matrix, the following relations can be obtained. It can be known from this relationship that the coordinate position of the self-propelled vehicle 1 can be obtained by conversion of the transformation matrix to obtain the coordinate position of the marker 6, that is, when the maximum movement error amount and the maximum rotation error amount of the self-propelled vehicle 1 When it is known, the entire area where the actual position of the self-propelled vehicle 1 is located can be transformed into the maximum range that the mark 6 may appear through the transformation matrix M, that is, the three-dimensional space before and after the virtual plane area ( An example is shown in Figure 4).

Because the robot arm 2 of the self-propelled device has different accuracy requirements for the operation of the workpiece 7, for example, when the accuracy requirement is higher, the image capturing unit 3 can capture the images on the virtual plane area. The area of the image area will be relatively small. Therefore, with different accuracy requirements, the number of the predetermined areas included in the virtual plane area is equal to or related to the ratio of the size of the virtual plane area and the size of each image in the virtual plane area. .

Referring again to FIG. 4, FIG. 4 is a schematic diagram illustrating the virtual plane area and the predetermined areas by way of example. For example, the virtual plane area P1 includes nine predetermined areas Z1 to Z9. When the self-propelled vehicle 1 moves to the actual position O2, the perspective of the image capturing unit 3 facing the actual direction can be in the predetermined area. Get the first image on Z5. When the mark 6 cannot be recognized in the first image, the self-propelled vehicle 1 controls the movement of the robot arm 2 so that the image capturing unit 3 is in the other predetermined areas Z1 to Z4 and Z6 in the virtual plane area P1. In ~ Z9, multiple other images are captured. As for the acquisition sequence of these other images, that is, controlling the relationship of the movement order of the robot arm 2, for example, toward the relative spiral positions of the predetermined areas Z4, Z1, Z2, and Z6 in sequence, or toward the predetermined areas Z1 to Z3 in sequence. Carpet scanning positions, etc. are not limited.

In addition, it should be added that in FIG. 4, because the lens of each image capturing unit 3 has a limitation of its depth of field, the mark 6 can only be used in a three-dimensional space with a certain depth before and after the virtual plane area P1. Only then can the image recognize the relative positional relationship between the mark 6 and the image capturing unit 3. For convenience of drawing, FIG. 4 only illustrates the three-dimensional space of the virtual plane area P1 away from the self-propelled vehicle 1, and the three-dimensional space of the virtual plane area P1 adjacent to the self-propelled vehicle 1 is omitted and not shown.

In step S4, the self-propelled vehicle 1 obtains the error amount according to the relative position relationship, so as to control the robot arm 2 to perform the gripping or moving operation on the workpiece 7. For example, in the case where the error amount is equal to zero, the relative position relationship and distance between the self-propelled cart 1 and the workpiece 7 are respectively T and O1 and D1 in FIG. 3, and when the error amount is greater than zero, The self-propelled vehicle 1 can obtain the relative positional relationship and distance between the workpiece 7 and the self-propelled vehicle 1 through image recognition, as shown in T and O2 and D2 of FIG. 3, respectively, and then make positioning compensation, so that The robot arm 2 is correctly controlled to correctly operate the workpiece 7.

In summary, the image captured by the self-propelled vehicle 1 to the image capturing unit 3 is used for image recognition to obtain the space between the workpiece 7 (or the mark 6) and the image capturing unit 3. The relative positional relationship between the two can further obtain the error amount of the predetermined position O1 and the actual position O2, so as to compensate and control the robot arm 2 to correctly move the workpiece 7. In addition, when the single image cannot be successfully identified, the self-propelled vehicle 1 is used to control the movement of the robot arm 2 to obtain another updated image and perform image recognition again until the identification is successful. Therefore, a technology using optical tracking can be realized to improve the positioning accuracy of the self-propelled device, so the objective of the invention can be achieved.

However, the above are only examples of the present invention. When the scope of implementation of the present invention cannot be limited by this, any simple equivalent changes and modifications made according to the scope of the patent application and the contents of the patent specification of the present invention are still Within the scope of the invention patent.

Steps S1 ~ S4‧‧‧‧

1‧‧‧Self-propelled car

2‧‧‧ robotic arm

3‧‧‧Image Acquisition Unit

6‧‧‧ mark

7‧‧‧ Workpiece

8‧‧‧ platform

D1‧‧‧distance

D2‧‧‧distance

O1‧‧‧ scheduled location

O2‧‧‧ actual location

T‧‧‧Location

X1‧‧‧ direction

X2‧‧‧ direction

Y1‧‧‧ direction

Y2‧‧‧ direction

dx‧‧‧movement error amount

dy‧‧‧movement error amount

dθ‧‧‧rotation error

P1‧‧‧Virtual plane area

Z1 ~ Z9 ‧‧‧ scheduled area

Other features and effects of the present invention will be clearly presented in the embodiment with reference to the drawings, in which: FIG. 1 is a flowchart illustrating an embodiment of a control method of a self-propelled device of the present invention; FIG. 2 is a schematic diagram To illustrate the positional relationship between a self-propelled equipment and a workpiece in this embodiment; FIG. 3 is a coordinate diagram exemplarily illustrating the relative relationship between a predetermined position and an actual position; and FIG. 4 is a schematic diagram, One virtual plane area and a plurality of predetermined areas are exemplarily described.

Claims (3)

A self-propelled device control method is suitable for a self-propelled vehicle, a robotic arm provided on the self-propelled vehicle, an image capture unit provided on the robotic arm, a workpiece at a target position, and an adjacent The marking of the workpiece, the control method includes the following steps: (a) receiving a driving signal by the self-propelled vehicle, and moving to an actual position, the actual position has an error amount relative to a predetermined position; (b) borrowing An image is captured by the image capture unit in an actual direction. When the self-propelled vehicle is at the predetermined position, the actual direction is pointed by the image capture unit to the workpiece, and the actual direction is by the image capture unit. Point to the mark; (c) image recognition of the image by the self-propelled vehicle to obtain the relative position relationship between the workpiece and the image capture unit, the self-propelled vehicle makes the mark in the image Image recognition to obtain the relative position relationship. When the self-propelled vehicle cannot identify the mark in the image, the self-propelled vehicle controls the robotic arm to move until the self-propelled vehicle controls the image. In the analysis of the images captured by the acquisition unit, until the mark is recognized, the self-propelled vehicle controls the movement of the robotic arm, so that the image acquisition unit acquires these images in turn for a plurality of predetermined areas, defining a A virtual plane area, the normal direction of the virtual plane area is parallel to the actual direction, the virtual plane area includes the predetermined areas, and one of the predetermined areas of the virtual plane area passes the position of the marker, the The size of the virtual plane area is related to a maximum movement error amount, a maximum rotation error amount, and a distance between the target position and the predetermined position of the self-propelled vehicle; and (d) by the self-propelled vehicle according to the relative Positional relationship to obtain the amount of error to control the robotic arm. The method for controlling a self-propelled device according to claim 1, wherein in step (c), the number of the predetermined areas included in the virtual plane area is related to the size of the virtual plane area and each image in The ratio of the size corresponding to the virtual plane area. The method for controlling a self-propelled device according to claim 2, wherein, in step (c), the mark may be a Chessboard map, a two-dimensional barcode (QR Code), a bar code (Bar Code), And either an Aruco code.