TWM606382U - Robotic manipulator system for real-time correction of clamping coordinate - Google Patents

Abstract

A mechanical arm system for real-time correction of clamping coordinates, comprising: a conveying platform, a mechanical arm, and a depth-of-field camera. The depth-of-field camera shoots an image frame of an object at the initial coordinate, and the position of the object is obtained by analyzing the image frame by the calculation unit , The center coordinates and the size ratio, and then calculate the clamping coordinates according to the center coordinates of the object, and the displacement path from the starting coordinates to the clamping coordinates. The calculation unit generates correction coordinates on the displacement path. When the claw is displaced to the calibration coordinates, take another image of the object to analyze and determine whether the clamping coordinates are consistent, so that the clamping claw can be accurately positioned to move to the clamping coordinates, that is, the depth of field camera can be used at the clamping claw The unobstructed image screen and the depth-of-field ranging function can make fine adjustments and corrections to the gripper in real time.

Description

Robotic arm system for real-time correction of clamping coordinates

This creation is related to a robotic arm control system, especially a robotic arm system that uses a depth-of-field camera to accurately control the position of the object and correct the clamping coordinates in real time.

By the way, in order to reduce manufacturing costs, conventional factories usually develop in the direction of automated manufacturing, and automated manufacturing methods are mainly realized through robotic arms. When existing robotic arms are operating, they usually choose appropriate grippers based on the actual manufacturing process. Wait for the retrieval mechanism, and the robotic arm can only perform repeated actions on a single type of object. In order to reduce the clamping error rate, the position and angle of the object must be strictly restricted. When the size of the object is not consistent or not touched When the rows are neatly arranged, the general mechanical arm that runs the fixed clamping program is not suitable. Therefore, the conventional art provides a mechanical arm combined with image analysis. This is because a fixed photographing device is installed in the working area of the mechanical arm. The device shoots an image of an object, and uses the image feature to control the robotic arm to produce corresponding adjustments. However, it is not difficult to find that there are still some shortcomings in the above-mentioned conventional structure. The main reasons are as follows: the camera is in a fixed position and cannot Through different distances or angles to verify the image characteristics of the object multiple times, when the shot When the object cannot be clamped correctly due to the image error, the robotic arm must be returned to the original position so that the robotic arm will not block the photographing device from shooting the object. The operation process that cannot be corrected in real time has time-consuming and low clamping accuracy. Disadvantages. In some cases that are difficult to clamp, the single-image method of judging an object after clamping fails, no matter how many subsequent re-clamping, the result will still be unable to successfully clamp the object. In summary The technical problems that the creation wants to improve.

In view of this, the creator has been engaged in the manufacturing, development and design of related products for many years, and after detailed design and careful evaluation of the above goals, he finally got a practical creation.

The technical problem to be solved by this creation is to provide a mechanical arm system for real-time correction of the clamping coordinates in response to the above-mentioned deficiencies in the prior art.

A conveying platform is used to continuously convey objects to a specified position. A mechanical arm is arranged on one side of the conveying platform, and the end of the mechanical arm is provided with a gripper that can move in three-dimensional coordinates, and the mechanical arm is built in for real-time judgment. A control unit of the jaw coordinates, and the fixed position of the jaw above the conveying platform is defined as the starting coordinate. A depth camera is fixed on the jaw of the robotic arm, and the front of the lens of the depth camera is aligned with the The gripper is oriented, and there is a calculation unit connected to the depth camera and the control unit. The depth camera shoots an image frame of the object at the start coordinate, and the position of the object is obtained by analyzing the image frame through the calculation unit, and is based on the start coordinate and Depth of field distance can calculate the object A center coordinate and size ratio of.

The depth-of-field camera includes a main lens and a pair of lenses, and the auxiliary lens is used for sensing distance, so that the image captured by the main lens has a depth effect.

The calculation unit is connected to a cloud server, and the cloud server stores the image screen used to identify the object and related data of the corresponding object, so that the calculation unit can deeply learn the image screen of the cloud server, thereby improving the objectivity. The accuracy of judgment.

When the depth-of-field camera executes the shooting command, the arithmetic unit can record the image frames of all objects, and create an image project folder on the cloud server, so that the subsequent images taken by the depth-of-field camera can directly compare the image project data The image frames in the folder can improve the calculation speed and judgment accuracy of the calculation unit.

When the gripper performs a displacement action along the displacement path, the calculation unit can record all the displacement paths and whether they reach the specified calibration coordinates or clamping coordinates, and create a path project folder on the cloud server, and judge the same through data analysis When neither of the similar displacement paths allows the gripper to reach the specified position, the calculation unit will subsequently eliminate the bad displacement path and improve the positioning accuracy by detouring.

The calculation unit uses the known starting coordinates and the depth of field of the image frame to perform calculations so that the image frame can draw a plane bounding box surrounding the object, and the plane bounding box is defined as a boundary coordinate at one end corner, In addition, the calculation unit uses the plane bounding box to calculate the center coordinates and the size ratio of the object.

The aspect ratio of the plane bounding box is set to (w, h), the boundary coordinates are set to (Xi, Yi), and the calculation unit uses the plane bounding box to calculate that the center coordinates are ((Xi-w/2 ),(Yi-h/2)), and the starting coordinates are set to (X ₀ ,Y ₀ ), and the size ratio of the object is set to Z, the X point movement vector of the corresponding object can be obtained as -((Xi -w/2)-X ₀ )/Z, and the Y point movement vector of the corresponding object is -((Yi-h/2)-Y ₀ )/Z.

The clamping jaws are displaced to the clamping coordinates so that the rotation angle of the clamping jaws is aligned with the center coordinates of the object. When the clamping jaws are insufficient or exceed the allowable value of the specified width, the calculation unit determines that the clamping has failed, and the control is The unit displaces the clamping jaws to the correction coordinates of the previous step, and takes the shot again with the depth-of-field camera, and the calculation unit recalculates the new clamping coordinates and performs the clamping operation again.

The mechanical arm is equipped with a load unit at the clamping jaw. After the clamping jaw is operated to perform a clamping action, the load unit does not detect that the clamping jaw maintains an increase in weight, and the calculation unit can judge the clamping If it fails, the control unit displaces the clamping jaw to the calibration coordinates of the previous step, and then takes the shot again by the depth camera, and the calculation unit recalculates the new clamping coordinates and performs the clamping action again.

When the clamping jaw fails to clamp the object for the number of times set by itself, the control unit operates the clamping jaw to rotate along the vertical axis of the conveying platform Rotate 90 degrees to clamp different positions of the object, or tilt the clamping jaws to allow the depth-of-field camera to additionally capture the side image of the object, thereby providing new clamping coordinates by analyzing the position of the center of gravity of the object.

The main purpose of this creation is that the depth-of-field camera shoots an image frame of the object at the starting coordinates, analyzes the image frame through the arithmetic unit to obtain the object position, center coordinates and size ratio, and then calculates the clamping according to the center coordinates of the object The coordinates and the displacement path from the starting coordinates to the clamping coordinates. The calculation unit generates correction coordinates on the displacement path. When the clamping jaws are displaced to the correction coordinates, an image of the object is taken again for analysis and judgment Whether the gripping coordinates are consistent, so that the gripper can be accurately positioned to move to the gripping coordinate, that is, the unobstructed image of the depth-of-field camera at the gripper and the depth-of-field ranging function can be used to fine-tune the gripper in real time Correction.

Other purposes, advantages, and novel features of this creation will be more apparent from the following detailed description and related drawings.

[This creation]

10: Conveying platform

20: Robotic arm

21: Gripper

22: control unit

23: Load cell

30: Depth of Field Camera

31: Calculation unit

32: main lens

33: Secondary lens

34: plane bounding box

341: boundary coordinates

35: Cloud server

A: Center coordinates

[Picture 1] is a three-dimensional view of this creation.

[Picture 2] is the lens diagram of the creative depth-of-field camera.

[Picture 3] is a schematic diagram of the image analysis of this creation.

[Figure 4] is a block diagram of the relationship between the components of this creation.

[Picture 5] is the flow block diagram of the action steps of this creation.

In order to enable your reviewer to have a further understanding and understanding of the purpose, features and effects of this creation, please cooperate with the following [Schematic Description] as detailed below:

First, please observe from the first, second and fourth figures, a robotic arm system for real-time correction of clamping coordinates, including: a conveying platform 10, a robotic arm 20, a depth camera 30, and a conveying platform 10 is used to continuously convey objects to a designated position. A robotic arm 20 is arranged on the side of the conveying platform 10, and the end of the robotic arm 20 is provided with a gripper 21 that can move in three-dimensional coordinates. A control unit 22 determines the coordinates of the gripper 21, and the fixed position of the gripper 21 above the conveying platform 10 is defined as the starting coordinate, a depth camera 30 is fixed on the gripper 21 of the robotic arm 20, and The front of the depth-of-field camera 30 is aligned with the jaw 21, and the depth-of-field camera 30 is connected to the control unit 22 with an arithmetic unit 31. The depth-of-field camera 30 includes a main lens 32 and a secondary lens 33. The secondary lens 33 is used for sensing distance, so that the image captured by the main lens 32 has a depth effect.

In actual use, please refer to Figures 4 and 5 in conjunction with Figure 1. When the object is conveyed by the conveying platform 10 to the corresponding position of the robotic arm 20, the depth-of-field camera 30 is aligned at the starting coordinates. Object shooting image screen. If no object is captured or the captured object is incomplete, the shooting will be repeated after a short interval. If an object is captured, the calculation unit 31 analyzes the image screen to know the location of the object , And based on the starting coordinates and the depth of field distance, you can calculate a center coordinate A and the size ratio of the object, and then according to the center of the object The standard A calculation knows the clamping coordinates. The clamping coordinates are the appropriate places above the object, so that the gripper 21 can clamp the center of gravity of the object, and then further plan the displacement path from the starting coordinates to the clamping coordinates , Basically let the gripper 21 move vectorially along the displacement path. Moreover, the calculation unit 31 automatically generates at least one correction coordinate on the displacement path. The correction coordinate is any point on the displacement path, which depends on the path length or path. Time-divided multi-segment correction coordinates, so that the gripper 21 can continuously and autonomously correct the coordinate position during the movement. When the control unit 22 moves the gripper 21 to the correction coordinate, the control unit 22 can first determine the gripper 21. When the error between the calibration coordinates and the actual position coordinates is too large, the control unit 22 moves the gripper 21 back to the starting coordinates and restarts. If there is only a slight error, the control unit 22 can directly Move the gripper 21 to the correct calibration coordinates, and let the depth-of-field camera 30 take an image of the object again, analyze the image through the arithmetic unit 31 and compare whether it is consistent with the previously determined gripping coordinates. The clamping jaw 21 moves to the next calibration coordinate or clamping coordinate, so that the clamping jaw 21 is accurately positioned and moved to the clamping coordinate, and then the clamping jaw 21 is clamped to a specified width based on the size ratio of the object, which can be used The unobstructed image of the depth-of-field camera 30 at the gripper 21 and the depth-of-field distance measurement function can perform fine adjustment and correction on the gripper 21 in real time, so as to achieve the effect of stably gripping objects.

To further explain its calculation method, please observe it from Figures 3 and 4. The calculation unit 31 uses the known starting coordinates and the depth of field of the image frame to perform calculations, so that the image frame can draw an image surrounding the object. A plane bounding box 34, and the plane bounding box 34 is defined as a boundary coordinate 341 at one end corner, and the calculation unit 31 uses the plane bounding box 34 to calculate the center coordinate A and the size ratio of the object, and the plane bounding box 34 The aspect ratio of is set to (w, h), the boundary coordinate 341 is set to (Xi, Yi), the calculation unit 31 uses the plane boundary box 34 to calculate that the center coordinate A is ((Xi-w/2) ,(Yi-h/2)), and the starting coordinates are set to (X ₀ ,Y ₀ ), and the size ratio of the object is set to Z, the X point movement vector of the corresponding object can be obtained as -((Xi- w/2)-X ₀ )/Z, and the Y point movement vector of the corresponding object is -((Yi-h/2)-Y ₀ )/Z, where the movement vector between X point and Y point is represented by a negative number, It is mainly the correspondence relationship between the displacement direction and the vector. It is not limited to the use of positive or negative numbers. In addition, the movement vector of point X and Y can also be multiplied and divided by the unit conversion value, so that the value of the movement vector can be amplified and read directly by the control unit 22 And application, the coordinate points of the image frame are converted into actual coordinate points through the above calculation process, and then the actual movement vector is further calculated and sent to the control unit 22 of the robotic arm 20 to achieve the purpose of driving the robotic arm 20.

Further explain the clamping action mechanism of the clamping jaw 21. As shown in Figures 4 and 5, after the clamping jaw 21 is displaced to the clamping coordinate, the rotation angle of the clamping jaw 21 is aligned with the center coordinate A of the object. And the control unit 22 drives the clamping jaw 21 to perform a clamping action. When the clamping jaw 21 is not enough or exceeds the allowable value of the specified width, the calculation unit 31 determines that the clamping fails, and the control unit 22 The clamping jaw 21 is displaced to the calibration coordinates of the previous step, and the depth camera 30 is used for shooting again, and the calculation unit 31 recalculates the new clamping coordinates and performs the clamping operation again. In another embodiment, the robotic arm 20 is positioned at the jaw 21 A loading unit 23 is installed. After the clamping jaw 21 is operated to perform a clamping action, the loading unit 23 does not detect that the clamping jaw 21 maintains an increase in weight. The calculation unit 31 can then determine that the clamping has failed. The control unit 22 displaces the clamping jaw 21 to the correction coordinates of the previous step, and the depth camera 30 takes pictures again, and the calculation unit 31 recalculates the new clamping coordinates and performs the clamping operation again. If the object is not successfully clamped by the above two methods, or the clamping jaw 21 fails to clamp the object for the number of times set by itself, the control unit 22 operates the clamping jaw 21 along the conveying platform 10 The vertical axis rotates 90 degrees to clamp different positions of the object, or tilt the clamping jaw 21 to make the depth-of-field camera 30 additionally capture the side image of the object, thereby providing new clamping coordinates for the object height analysis of the center of gravity position After the object is successfully clamped, the control unit 22 controls the clamping jaw 21 to return to the initial coordinate position, and waits for the next object to enter the movable range of the robotic arm 20.

Another embodiment of this creation, please look at it as shown in Figures 4 and 5. The calculation unit 31 is connected to a cloud server 35, and the cloud server 35 stores image frames and corresponding objects used to identify objects. The relevant data allows the calculation unit 31 to deeply learn the image frame of the cloud server 35, thereby improving the accuracy of determining the object. On the other hand, when the depth-of-field camera 30 executes a shooting command, the arithmetic unit 31 can record the image frames of all objects, and create an image project folder on the cloud server 35 to enable the depth-of-field camera 30 to subsequently capture the image frames Can directly compare the images in the image project folder, and improve the calculation through a large amount of self-deep learning process The calculation speed and judgment accuracy of the unit 31. Furthermore, when the gripper 21 performs a displacement action along the displacement path, the arithmetic unit 31 can record all the displacement paths and whether the specified calibration coordinates or clamping coordinates are reached, and create a path project folder on the cloud server 35, When it is judged through data analysis that the same or similar displacement paths cannot allow the gripper 21 to reach the specified position, it is likely to be a long-standing mechanical defect of the mechanical arm 20, which causes the gripper 21 to detach every time it moves to a fixed coordinate position. Displacement path. At this time, the calculation unit 31 will subsequently eliminate the bad displacement path. When the newly generated displacement path overlaps the excluded bad displacement path, a new path is re-established on the side of the path, that is, the positioning is improved by detouring The accuracy is to improve the displacement efficiency of the robot arm 20.

To sum up, this creation has indeed achieved a breakthrough structural design, and has improved new content, while at the same time it can achieve industrial utilization and progress. Moreover, this creation has not been seen in any publications, and it is also novel. In accordance with the relevant provisions of the Patent Law, Yan filed a new patent application in accordance with the law, and I implore the Jun Bureau review committee to grant legal patent rights. I really pray.

Only the above is only one of the preferred embodiments of this creation, and should not be used to limit the scope of implementation of this creation; that is, all equal changes and modifications made in accordance with the scope of the patent application for this new model should still belong to this new model Covered by the patent.

10: Conveying platform

20: Robotic arm

21: Gripper

30: Depth of Field Camera

Claims (10)

A mechanical arm system for real-time correction of clamping coordinates, including: A conveying platform, which is used to continuously convey objects to a designated position; A mechanical arm is arranged on one side of the conveying platform, and the end of the mechanical arm is provided with a gripper that can move in three-dimensional coordinates, and a control unit for real-time judgment of the coordinate of the gripper is built in the mechanical arm, and the gripper The fixed position of the claw above the conveying platform is defined as the starting coordinate; A depth-of-field camera, which is fixed on the gripper of the robotic arm, the front of the lens of the depth-of-field camera is aligned with the direction of the gripper, and the depth-of-field camera is connected to the control unit with an arithmetic unit; The depth-of-field camera shoots an image frame of the object at the starting coordinates, and the calculation unit analyzes the image frame to know the position of the object, and based on the starting coordinates and the depth-of-field distance, it can calculate a center coordinate and size ratio of the object. The center coordinate calculation of the object obtains the clamping coordinates and the displacement path from the starting coordinate to the clamping coordinates. The calculation unit automatically generates at least one correction coordinate on the displacement path. When the control unit moves the clamping jaw to When the coordinates are calibrated, the depth-of-field camera again shoots an image frame of the object, analyzes the image frame through the calculation unit and compares whether it is consistent with the previously determined clamping coordinates, so that the clamping jaw can be accurately positioned and moved to the clamping coordinates , And then clamp the gripper to a specified width based on the size ratio of the object, that is, the unobstructed image frame of the depth-of-field camera at the gripper and the depth-of-field ranging function can be used to fine-tune and correct the gripper instantly. For example, the robotic arm system for real-time correction of the clamping coordinates of claim 1, wherein the depth-of-field camera includes a main lens and a sub-lens, and the sub-lens is used to sense the distance so that the image captured by the main lens has a depth effect. For example, the robotic arm system for real-time calibration of the clamping coordinates of the request item 1, wherein the calculation unit is connected to a cloud server, and the cloud server stores the image screen used to identify the object and the related data of the corresponding object, so that the calculation unit Deeply learn the image of the cloud server to improve the accuracy of the judgment of the object. For example, the robotic arm system for real-time calibration of the clamping coordinates of claim 3, where the depth-of-field camera executes the shooting command, the arithmetic unit can record the image frames of all objects, and create an image project folder on the cloud server to make the The subsequent image frames captured by the depth-of-field camera can be directly compared with the image frames in the image project folder, thereby improving the calculation speed and accuracy of the calculation unit. For example, the mechanical arm system for real-time correction of the clamping coordinates of claim 3, wherein when the clamping jaw performs a displacement action along the displacement path, the calculation unit can record all the displacement paths and whether it reaches the designated correction coordinates or the clamping coordinates, and then The cloud server creates a path project folder, and when data analysis determines that the same or similar displacement paths cannot allow the gripper to reach the specified position, the calculation unit will subsequently eliminate the bad displacement path and improve the positioning accuracy by detouring. For example, the mechanical arm system for real-time correction of the clamping coordinates of claim 1, wherein the calculation unit uses the known starting coordinates and the depth of field of the image frame to perform calculations, so that the image frame can draw a plane bounding box surrounding the object, And the plane bounding box is defined as a boundary coordinate at one end corner, and the calculation unit calculates the center coordinate and the size ratio of the object based on the plane bounding box. For example, the robot arm system for real-time correction of the clamping coordinates of claim 6, wherein the aspect ratio of the plane bounding box is set to (w, h), the boundary coordinates are set to (Xi, Yi), and the calculation unit is based on the plane boundary Box operation knows that the center coordinate is ((Xi-w/2),(Yi-h/2)), and the starting coordinate is set to (X ₀ ,Y ₀ ), and the object size ratio is set to Z, That is, the X point movement vector of the corresponding object is -((Xi-w/2)-X ₀ )/Z, and the Y point movement vector of the corresponding object is -((Yi-h/2)-Y ₀ )/ Z. For example, the mechanical arm system for real-time correction of the clamping coordinates of request 1, in which the clamping jaw is displaced to the clamping coordinate, so that the rotation angle of the clamping jaw is aligned with the center coordinate of the object, and the clamping jaw is insufficient or exceeds the specified width When the allowable value is allowed, the arithmetic unit judges that the clamping has failed, the control unit moves the gripper to the calibration coordinates of the previous step, and the depth-of-field camera shoots again, and the arithmetic unit recalculates the new clamping coordinates. Perform clamping action. For example, the mechanical arm system for real-time correction of the clamping coordinates of claim 1, wherein the mechanical arm is equipped with a load unit at the gripper, and the operation After the clamping jaw is used for the clamping action, the load unit does not detect that the clamping jaw maintains an increase in weight, the calculation unit can then judge the clamping failure, and the control unit displaces the clamping jaw to the previous step of calibration The coordinates are photographed by the depth-of-field camera again, and new clamping coordinates are recalculated by the calculation unit, and the clamping action is performed again. For example, the mechanical arm system for real-time correction of the clamping coordinates of claim 1, where the gripper fails to grip the object for the number of times set by itself, the control unit operates the gripper to rotate 90 along the vertical axis of the conveying platform In this way, different positions of the object can be clamped, or the clamping jaw can be tilted so that the depth-of-field camera additionally captures the side image of the object, thereby providing new clamping coordinates by analyzing the position of the center of gravity of the object.

Images