WO2022222513A1

WO2022222513A1 - Method and apparatus for filling grooves on basis of controlling moving speed of robot

Info

Publication number: WO2022222513A1
Application number: PCT/CN2021/138498
Authority: WO
Inventors: 李辉; 魏海永; 丁有爽; 邵天兰
Original assignee: 梅卡曼德（北京）机器人科技有限公司
Priority date: 2021-04-20
Filing date: 2021-12-15
Publication date: 2022-10-27
Also published as: CN112967307A

Abstract

Provided are a method and apparatus for filling grooves on the basis of controlling the moving speed of a robot. The method for filling grooves on the basis of controlling the moving speed of a robot comprises: extracting a groove contour of the surface of an object; according to a preset trajectory point interval, traversing points on the contour, and determining a trajectory point for robot movement at each traversed point; on the basis of the groove width Wk at the trajectory point, determining the robot moving speed Vk at the trajectory point; and executing groove filling according to the determined trajectory points and the moving speeds at the trajectory points.

Description

Groove filling method and device based on robot movement speed control

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of the Chinese patent application number "202110426174.4", which was filed on April 20, 2021, and the application name is "Groove Filling Method, Device, Electronic Device and Storage Medium Based on Robot Movement Speed Control", The entire content of this Chinese patent application is incorporated herein by reference.

technical field

The present application relates to the field of B25 intelligent robots, and more particularly, to a method and device for filling grooves based on robot movement speed control.

Background technique

At present, with the widespread popularization of intelligent program-controlled robots, the operation of filling fillers in grooves on the surface of objects has been realized with the help of intelligent program-controlled robots. However, conventional intelligent robots can only fill in fixed types of items and fixed scenes. In this case, the robot must fill along a fixed trajectory with a fixed moving speed and discharge speed. This method cannot be applied to fixed Items and industrial scenes other than fixed scenes. Even with the help of the existing robot vision technology to identify the grooves on the surface of different items and determine the filling trajectory for filling, it is difficult to achieve accurate filling without shortage or accumulation. On the one hand, the difficulty is that all grooves cannot be accurately identified. On the other hand, the difficulty lies in that if the grooves with different characteristics are filled with the same filling rules, the problem of inconsistent filling degree of the filler in the grooves is easy to occur.

technical solutions

In view of the above-mentioned problems, the present application is made to overcome the above-mentioned problems or at least partially solve the above-mentioned problems. Specifically, one of the innovations of the present application is that the applicant finds that one of the reasons for the above-mentioned inconsistency in filling degree is that the prior art does not provide different filling methods for different grooves, for example, when the grooves are linear , the robot must move while filling, and when the groove is a point, the robot must accurately identify the location of the point and fill at that location. Therefore, the present application proposes a method of first identifying the groove type, and then letting the robot use the filling scheme corresponding to the type to fill, thereby greatly improving the accuracy of groove filling.

The second innovation of the present application is that the applicant found that the reason why it is difficult to accurately identify all the grooves is that the existing robot vision technology uses the three-dimensional point cloud information of the object, that is, the three-dimensional image information for identification. It is shallow, and the groove cannot be identified according to the 3D point cloud information. Therefore, the present application can accurately identify and extract shallow grooves in combination with the two-dimensional image by acquiring a two-dimensional image of the surface of the object, and performing equalization and segmentation operations on the two-dimensional image, which greatly improves the ability of groove identification.

The third innovation of the present application is that the applicant found that another reason for the above-mentioned problem of inconsistent filling degree is that in the prior art, the moving speed of the robot is fixed, and the discharging speed is also fixed. When narrow, the robot performs filling at an average moving speed and discharge speed, so wide grooves cannot be filled, and a buildup of filler can form in shallow grooves. Therefore, the present application adjusts the moving speed of the robot for different groove widths without changing the discharge speed, thereby greatly improving the accuracy of groove filling.

The fourth innovation of the present application is that the applicant found that although the grooves are both line-shaped, the characteristics of closed grooves and non-closed grooves are completely different, and the same filling scheme cannot be used. In particular, for non-closed grooves, there are The defined open end, the starting point for optional filling like closed grooves, requires the development of a dedicated filling method for non-closed grooves. Therefore, according to the characteristics of the non-closed grooves, the present application proposes a filling method specially used for the non-closed grooves on the surface of the object, thereby greatly improving the filling accuracy of the non-closed grooves.

All solutions disclosed in the claims and descriptions of the present application have one or more of the above-mentioned innovations, and correspondingly, can solve one or more of the above-mentioned technical problems. Specifically, the present application provides a groove filling method based on robot movement speed control, a groove filling device based on robot movement speed control, an electronic device and a computer-readable storage medium.

The groove filling method based on the robot movement speed control according to the embodiment of the present application includes:

Extract the contour of the groove on the surface of the object;

Traverse the points on the contour according to the preset trajectory point interval, and determine the trajectory point of the robot movement at each traversed point;

Determine the robot moving speed Vk at the track point based on the groove width Wk at the track point;

The groove filling is performed according to the determined track point and the movement speed at the track point.

In certain embodiments, extracting the surface groove profiles of the object includes removing non-groove profiles.

In some embodiments, the removing the non-groove contour includes removing the non-groove contour according to the number of pixels.

In some embodiments, the method further includes: presetting a moving speed V0 corresponding to a specific width W0, and calculating the moving speed Vk of the robot at the trajectory point according to W0, V0 and Wk.

In some embodiments, it also includes: the moving speed Vk at the trajectory point is a linear or nonlinear ratio of V0 and Rk, where Rk=Wk/W0.

In certain embodiments, the groove profile includes an inner profile and an outer profile.

In some embodiments, the determining the trajectory point of the robot movement according to the traversed points specifically includes: at each traversed point, obtaining the point along the local normal direction to the corresponding outer contour or inner contour The point between the point and the corresponding point on the outer or inner contour is selected as the trajectory point.

In some embodiments, the trajectory point is the midpoint between the traversed point and the corresponding point on the outer or inner contour.

The groove filling device based on the robot movement speed control according to the embodiment of the present application includes:

The contour extraction module is used to extract the groove contour on the surface of the object;

The trajectory point determination module is used to traverse the points on the contour according to the preset trajectory point interval, and determine the trajectory point of the robot moving at each traversed point;

a movement speed determination module, configured to determine the robot movement speed Vk at the track point based on the groove width Wk at the track point;

The filling module is used to perform groove filling according to the determined trajectory point and the movement speed at the trajectory point.

An electronic device according to an embodiment of the present application includes a memory, a processor, and a computer program stored on the memory and executable on the processor, and the processor implements any of the above-mentioned embodiments when the processor executes the computer program. Groove filling method based on robot movement speed control.

The computer-readable storage medium of the embodiment of the present application has a computer program stored thereon, and when the computer program is executed by the processor, realizes the groove filling method based on the movement speed control of the robot of any of the above-mentioned embodiments.

Additional aspects and advantages of the present application will be set forth, in part, from the following description, and in part will become apparent from the following description, or may be learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic flowchart of a groove filling method based on groove identification according to some embodiments of the present application;

2 is a schematic flowchart of a method for extracting a shallow groove profile on an object surface according to some embodiments of the present application;

3 and 4 are schematic diagrams of groove contour identification and extraction according to some embodiments of the present application;

5 is a schematic flowchart of a groove filling method based on robot movement speed control according to some embodiments of the present application;

6 is a schematic flowchart of a method for filling a non-closed groove according to some embodiments of the present application;

7 is a schematic diagram of a non-closed groove filling method according to some embodiments of the present application;

8 is a schematic structural diagram of a groove filling device based on groove identification according to some embodiments of the present application;

9 is a schematic structural diagram of an object surface shallow groove contour extraction device according to some embodiments of the present application;

10 is a schematic structural diagram of a groove filling device based on robot movement speed control according to some embodiments of the present application;

11 is a schematic structural diagram of a non-closed groove filling device according to some embodiments of the present application;

FIG. 12 is a schematic structural diagram of an electronic device according to some embodiments of the present application.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

FIG. 1 shows a groove filling method according to an embodiment of the present application, including:

Step S100, extracting the contour of the groove on the surface of the object;

Step S200, identifying the type of groove profile;

Step S300, determining a corresponding groove filling scheme according to the type of groove profile;

Step S400, filling the grooves based on the groove filling scheme.

In step S100, the 3D point cloud information of the object can be obtained. For example, the point cloud information can be obtained through a 3D industrial camera. The 3D industrial camera is generally equipped with two lenses, which respectively capture the group of objects to be grasped from different angles. It can realize the display of three-dimensional images of objects. Place the group of objects to be grasped under the vision sensor, and shoot with two lenses at the same time. According to the obtained relative pose parameters of the two images, use a general binocular stereo vision algorithm to calculate the X, X, and X of each point of the object to be filled. The Y, Z coordinate values and the coordinate orientation of each point are then converted into the point cloud data of the item group to be grasped. During specific implementation, components such as laser detectors, visible light detectors such as LEDs, infrared detectors, and radar detectors can also be used to generate point clouds, and the application does not limit the specific implementation.

The point cloud data acquired in the above manner is three-dimensional data. In order to facilitate data processing and improve efficiency, the acquired three-dimensional point cloud data can be orthographically mapped onto a two-dimensional plane.

As an example, a depth map corresponding to the orthographic projection can also be generated. A two-dimensional color map corresponding to a three-dimensional object region and a depth map corresponding to the two-dimensional color map can be acquired in a depth direction perpendicular to the object. Wherein, the two-dimensional color map corresponds to an image of a plane area perpendicular to the preset depth direction; each pixel in the depth map corresponding to the two-dimensional color map is in one-to-one correspondence with each pixel in the two-dimensional color map, and each pixel The value of the pixel is the depth value of the pixel.

This application can be used for various groove filling scenarios in the industry, such as the scenario of gluing the grooves on the surface of the table board. In some scenes, the grooves on the surface of the object are shallow, and in the 3D point cloud or depth map, the point clouds of the inner and outer contours of the groove cannot be distinguished, so the contour of the groove cannot be identified from the 3D point cloud or depth map. In order to be able to extract the contours of such shallow grooves, the applicant has developed a method for extracting the contours of shallow grooves. In this method, combined with the acquired two-dimensional images, the two-dimensional images are processed to obtain a method that can clearly display the grooves. The two-dimensional image of the 2D image, so as to accurately identify and extract the groove contour, which is one of the focuses of this application.

FIG. 2 shows a method for extracting a shallow groove profile according to an embodiment of the present application, including:

Step S110, acquiring a two-dimensional image of the object surface;

Step S120, performing an image equalization operation on the image;

Step S130, performing an image segmentation operation on the equalized image;

Step S140: Determine the shallow groove according to the segmented image and extract the outline of the shallow groove.

In step S110, a two-dimensional image of the surface of the object is acquired. A three-dimensional image of the object, such as 3D point cloud information, can be acquired according to the method of S100, and then mapped into a two-dimensional image.

In step S120, an image equalization operation is performed on the acquired two-dimensional image. Due to the image equalization operation, the original grayscale with fewer pixels will be allocated to other grayscales, and the pixels are relatively concentrated. Image details that cannot be seen clearly. In the original two-dimensional image, due to the close contrast between the surface of the object and the shallow groove, it cannot be effectively identified. In order to solve this problem, an image equalization operation can be performed under the condition of ensuring a certain amount of illumination. After such processing, the shallow groove can be clearly identified. groove.

In step S130, an image segmentation operation is further performed on the two-dimensional image after performing the image equalization operation. After the contrast between the object surface and the shallow grooves is close to a certain level, the grooves may not be clearly identified even after image equalization. At this time, image segmentation can be performed on the two-dimensional image that has undergone the image equalization operation, and specifically, graphics with properties similar to shallow grooves in the image can be segmented and extracted. In one embodiment, after segmentation and extraction, a two-dimensional image as shown in FIG. 3 is formed. In the two-dimensional image, many shallow grooves are clearly visible.

In step S140, the shallow groove is determined according to the segmented image and the outline of the shallow groove is extracted. As shown in FIG. 3 , although all the grooves are obtained, many of the grooves are only pits on the surface of the object, and these pits may, for example, be generated by collision and do not need to be filled. In order to eliminate these grooves that do not need to be filled, a minimum pixel threshold can be preset, and grooves with a number of pixels less than the threshold are regarded as pits and do not need to be filled. In this way, when extracting the groove outline, for each groove, it is possible to first judge whether the number of pixels in the groove part is greater than the minimum pixel threshold, and if it is greater than the groove, it is considered that the groove needs to be filled, otherwise it is not required. Filled pits. In addition, a maximum pixel threshold can also be preset to screen out specific concave structures on the surface of the object. For example, some object surfaces may be provided with sub-packaging compartments for placing objects. In this way, only when the number of pixel points in the groove portion is less than the threshold, the groove is identified as a groove to be filled. In addition, after image equalization, image segmentation and other processing, in the formed two-dimensional image as shown in Figure 3, multi-layer grooves can be identified from the outside to the inside. For grooves located within the outermost grooves , which can be defined as the inner groove. These inner grooves, with clear outlines, are easy to identify and extract. However, for the outermost groove, only the inner contour can be clearly identified, and the outer contour is integrated with the black background, so the position of the outer contour cannot be accurately identified. In order to determine the outer contour, the position of the outer edge of the outermost groove, that is, the position of the outer contour, can be determined according to the previously acquired three-dimensional image, and then combined with the two-dimensional image to obtain the outer contour of the outermost groove .

In step S200, based on the extracted groove profile, the type of the groove profile is identified. In this application, in order to fill the grooves accurately, so that the filler can fill the grooves without overflowing from the grooves, so a variety of filling methods are designed, each filling method corresponds to a specific type of contour, this is the one of the main points of the application. The grooves can be classified according to the type of grooves that can be filled by the groove filling method used. In the embodiment shown in FIG. 4 , the grooves are divided into closed grooves, open grooves and single-point grooves. Roughly speaking, the outline of the single-point grooves is single-point, closed grooves and open grooves. The grooves are linear. Among them, the closed groove is in the shape of a closed line, and there is no clear starting point and ending point. The open grooves are in the shape of non-closed lines, with clear start and end points.

In step S300, a corresponding groove filling scheme is determined according to the type of groove profile. In a preferred embodiment of the present application, the groove types correspond to the filling methods one-to-one. In other implementation manners, multiple filling modes may also correspond to one type of groove, or multiple types of grooves may correspond to one filling mode, which is not limited in this application. The focus of this application is to call different filling schemes for different grooves, so any filling scheme can be used, which is not limited in this application. However, in a preferred embodiment, three different filling schemes, a closed groove filling method, an open groove filling method and a point groove filling scheme, are designed for closed grooves, open grooves and point grooves, respectively. filling.

In step S400, the grooves are filled based on the groove filling scheme. In this application, a robot can be used for groove filling. For this purpose, it is necessary to plan the movement trajectory, movement speed and filling speed of the robot first. The robot moves on the surface of the object according to the planned path and speed, and follows the planned filling speed. Fill the groove with filler. Three different filling schemes, the closed groove filling method, the open groove filling method and the point groove filling scheme, are described below.

single point groove

According to the acquired two-dimensional contour of the single-point groove, it can be mapped to a three-dimensional point cloud, and then the pose information of the three-dimensional point cloud can be obtained, and the robot can be moved to the position of the single-point groove for filling according to the pose information.

closed groove

When the robot fills the groove, it will control the discharge head to fill based on a certain discharge rate. The discharge rate, as an inherent property of the robot, affects the filling effect in this embodiment. Since the discharge rate of the robot is usually fixed, if the width of the groove to be filled is inconsistent on the entire path, and the robot moves at the same motion speed at each trajectory point, it may lead to wide grooves. There is a lack of glue at the groove, and there is a glue pile at the narrow groove. In order to solve this problem, this application proposes a method that controls the moving speed of the robot according to the width of the groove to control the discharge rate at the groove. scheme, which is also one of the focuses of this application.

Fig. 5 shows the groove filling method of the present application by controlling the moving speed of the robot. The method can be used for the filling of closed grooves and open grooves in the present application, and the method includes:

Step 411, extracting the contour of the groove on the surface of the object;

Step 412, traverse the points on the contour according to the preset trajectory point interval, and determine the trajectory point of the robot movement at each traversed point;

Step 413: Determine the robot moving speed Vk at the track point based on the groove width Wk at the track point;

Step 414 , let the robot perform groove filling according to the determined trajectory point and the movement speed at the trajectory point.

For step 411, a method similar to step S100 can be used to extract the groove contour on the surface of the object, which will not be repeated here.

For step 412, the closed groove contour consists of two contours, an inner and outer contour. In one embodiment, in order to obtain the trajectory of the robot motion, the interval of the trajectory points can be preset, and then the inner contour or the outer contour is used as a reference, and the points on the reference contour are traversed according to the interval of the trajectory points. The track point interval can be a distance, that is, a track point can be set to be generated every certain distance, in other words, that is, a point on the contour is traversed every certain distance, and the entire contour is traversed in this way. For each traversed point P1, find the point P2 corresponding to the current point P1 along the local normal direction to another contour. If the traversal is performed on the inner contour, P1 is on the inner contour and P2 is on the outer contour. The distance from P1 to P2 is the groove width W here. A point P3 can be selected between P1 and P2 as the fill track point. Preferably, the filling track point can be the midpoint of the groove width W, that is, the point is the position on the width W where the distance ratio to the inner and outer contours is 1:1. According to the actual filling needs, the positions of different distance ratios can also be selected as track points. For example, if a groove is deep on one side and shallow on the other side, then filling at the midpoint of a groove with a width of W, Since the cross-sectional area from this point to the inner contour is greater than its cross-sectional area to the outer contour, filler may accumulate outside the outer contour. At this time, in order to accurately fill the groove, the track point may be set at a position closer to the inner contour, for example, the distance between the track point and the inner and outer contour may be set to 2:3. In other embodiments, instead of using the ratio, the position of the track point may be adjusted by the distance, for example, the track point is set at a distance of 1 mm from the inner contour. After obtaining all the trajectory points of the robot on the two-dimensional plane in the above manner, and forming a complete moving trajectory, these two-dimensional trajectory points are used to obtain three-dimensional trajectory points through the 2D to 3D mapping relationship, and a three-dimensional moving trajectory has been formed.

For step 413, when the robot fills the groove, it will control the discharge head to fill based on a certain filling rate. The discharge rate, as an inherent property of the robot, affects the filling effect in this embodiment. Since the discharge rate of the robot is usually fixed, if the width of the groove to be filled is inconsistent on the entire path, and the robot moves at the same motion speed at each trajectory point, it may lead to wide grooves. Underfill occurs in the grooves and overflow occurs in the narrow grooves. In the case of a fixed discharge rate, different robot moving speeds will lead to different discharge amounts. Roughly speaking, when the discharge rate is fixed, the faster the robot moves, the less material will be discharged, and vice versa. more. Generally speaking, a wider groove needs to be filled with a larger amount of material, so the robot needs to move relatively slowly at the groove. In one embodiment, a suitable robot speed V0 corresponding to a specific width W0 may be predetermined. In this way, for each track point, the width Wk of the groove at the track point can be obtained first. If Wk>V0, the robot moving speed Vk at the track point should be less than V0, and vice versa.

For step 414, the ratio Rk=Wk/W0 of the groove width Wk to the specific width W0 can be calculated first, then the moving speed Vk at the current track point can be a linear or nonlinear ratio of V0 and Rk. In one embodiment, the movement speed of the robot at each trajectory point can be calculated for each trajectory point using the following formula:

Vk=V0/Rk.

open groove

Closed grooves are two contours. When setting track points, all points on the contour can be traversed from any point on the inner or outer contour according to the preset track point interval. Different from the closed groove, the open groove has a specific opening end point in shape, so one open end point must be used as the starting point, and the other open end point must be used as the end point, so that according to the preset trajectory point interval, from the opening point The contour is traversed from point to end point. Therefore, different from closed grooves, open grooves also need to determine the correct starting point and ending point of the trajectory to accurately form the movement trajectory of the robot. The applicant has made a lot of effort to design a groove filling solution that can be used for any groove with a defined start point and an end point, that is, a non-closed groove, which is also one of the focuses of this application.

Figure 6 shows the non-closed groove filling method of the present application, including:

Step 421, extract the contour of the non-closed groove on the surface of the object;

Step 422, generating a circumscribed rectangle of the outline;

Step 423, based on the generated circumscribed rectangle, determine two open end points of the outline;

In step 424, groove filling is performed using the two open end points as the starting point and the ending point, respectively.

FIG. 7 shows a schematic diagram of analyzing the opening and contour of a non-closed groove through a circumscribed rectangle according to an embodiment of the present application. The following describes the non-closed groove filling method of the present application with reference to FIG. 7 .

For step 421, a method similar to that of step S100 can be used to extract the contour of the groove on the surface of the object, which will not be repeated here.

For step 422, the minimum circumscribed rectangle is obtained for the outline of the opening groove, and any feasible method can be used to obtain the minimum circumscribed rectangle, which is not limited in this application;

For step 423, the end point of the opening is determined according to the minimum circumscribed rectangle. For example, according to the groove outline, it is determined that the opening point is located on a certain side of the circumscribed rectangle, and then the point on the outline closest to the side is determined as the opening end point. In a preferred embodiment, two open end points P1 and P2 can be determined.

For step 424, the entire groove contour is broken at P1 and P2 to form two separate contour segments S1 and S2. In the embodiment shown in FIG. 7, S1 is the outer contour and S2 is the inner contour. In this way, the starting point P1 and the ending point P2 of the traversal are obtained, as well as the outer contour S1 where P1 and P2 are located, and the inner contour S2 opposite to the outer contour. From P1 to P2, traversal sampling is performed on S1 according to the preset trajectory point interval. The track point interval can be a distance, that is, a track point can be set to be generated every certain distance, in other words, the contour points are traversed every certain distance. For each point P1 traversed, obtain the point P3 corresponding to the current point P1 along the local normal direction to another contour. The distance from P1 to P3 is the groove width W here. Pick a point P4 between P1 and P3, and use this point as the fill track point. Preferably, the filling track point may be the midpoint of the groove width W, that is, the point is a position on the width W that is 1:1 away from the inner and outer contours. According to the actual filling needs, the positions of different distance ratios can also be selected as trajectory points. For example, if the cross-sectional area from the midpoint of the width W to the inner contour is larger than the cross-sectional area to the outer contour, for accurate filling, you can Set the track point to be closer to the inner contour. For example, the distance between the track point and the inner and outer contour can be set to 2:3. In other embodiments, instead of using the ratio, the position of the track point may be adjusted by the distance, for example, the track point is set at a distance of 1 mm from the inner contour. After acquiring the complete movement trajectory of the robot on the two-dimensional plane in this way, these two-dimensional trajectory points are used to obtain three-dimensional trajectory points through a 2D-to-3D mapping relationship.

When the robot fills the groove, it will control the discharge head to fill based on a certain filling rate. The discharge rate, as an inherent property of the robot, affects the filling effect in this embodiment. Since the discharge rate of the robot is usually fixed, if the width of the groove to be filled is inconsistent on the entire path, and the robot moves at the same motion speed at each trajectory point, it may lead to wide grooves. Underfill occurs in the grooves and overflow occurs in the narrow grooves.

In the case of a fixed discharge rate, different robot moving speeds will lead to different discharge amounts. Roughly speaking, when the discharge rate is fixed, the faster the robot moves, the less material will be discharged, and vice versa. more. Generally speaking, a wider groove needs to be filled with a larger amount of material, so the robot needs to move relatively slowly at the groove. In one embodiment, a suitable robot speed V0 corresponding to a specific width W0 may be predetermined. In this way, for each track point, the width Wk of the groove at the track point can be obtained first. If Wk>V0, the robot moving speed Vk at the track point should be less than V0, and vice versa.

The ratio Rk=Wk/W0 of the groove width Wk to the specific width W0 can be calculated first, then the moving speed Vk at the current track point can be a linear or nonlinear ratio of V0 and Rk. In one embodiment, the movement speed of the robot at each trajectory point can be calculated for each trajectory point using the following formula:

Vk=V0/Rk.

According to the above embodiment, firstly, the present application can implement filling methods dedicated to different grooves, thereby greatly improving the filling accuracy; secondly, the present application can accurately identify the grooves to be filled even if the grooves to be filled are shallow. Thirdly, the application controls the movement rate of the robot according to the groove width, which can accurately fill the grooves with different widths and improves the filling accuracy; fourthly, for special opening grooves, the application also develops special The method can improve the filling accuracy of such grooves. It can be seen that the present application solves all the problems in the process of using the robot to fill the groove.

In addition, those skilled in the art can also make various changes and modifications to the above-mentioned embodiments:

The robots in various embodiments of the present application may be industrial robot arms, and these robot arms may be general-purpose or dedicated to groove filling. The application can fill the grooves on the surface of any object, such as glass, table board, steel plate, etc. The application can be filled with any filler, such as glue, various chemical fillers, etc. The application does not limit the specific application field, as the preferred This application is particularly suitable for applying glue in the grooves of the table top.

In order to make the robot take fewer redundant trajectories, the initial point of the trajectory point can be set at the position on the trajectory path that is closest to the initial pose of the robot, for example, the initial point is set in the middle of the edge close to the robot. That is to say, after the initial pose of the robot is determined, the middle point on the trajectory path of the side closest to the initial pose of the robot can be used as the initial point of the trajectory point, and then according to the inherent properties of the robot Other track points are set on the track path, and then the track point information can be obtained. It is worth mentioning that the track point information may include, but is not limited to, the coordinates of the track point, the initial track point of the track point, and the direction of the track point (ie, the position sequence of the track point). After obtaining the trajectory point information, the trajectory point information can be sent to the robot by means of communication. When the robot receives the trajectory point information, it can control its own spray nozzle to fill the groove based on the trajectory point information.

In some embodiments, according to the inherent properties of the robot and the initial pose of the robot, trajectory point information is generated on the trajectory path, including:

Determine the corners and straight lines in the trajectory path;

According to the discharge rate and movement speed of the robot, set the trajectory points at the corresponding density at the corners and straight lines;

According to the initial pose of the robot, determine the movement sequence of the trajectory points to obtain the trajectory point information.

Specifically, determining the corners and straight lines in the trajectory path may be determined based on the relationship between the coordinate values of each point on the trajectory path. The X and Y coordinates of the adjacent points at the corner will be different, while the adjacent points at the straight line may have the same X coordinate or the same Y coordinate. For example: Assuming that the shape of the item to be filled is a rectangle, in the trajectory path of the item to be filled, the X and Y coordinates of the adjacent points at the corners of the four corners will be different, and the Y coordinates of the adjacent points at the upper line will be different. The coordinates will be the same but the X-coordinates will be different, the Y-coordinates of the adjacent points on the lower line will be the same but the X-coordinates will be different. The Y coordinate will be different, the X coordinate of the adjacent points on the right line will be the same, but the Y coordinate will be different, and the X coordinate will be smaller than the value of the left line.

When the robot is filling, it will control the discharge head to fill based on a certain discharge rate. As an inherent property of the robot, the discharge rate affects the filling effect in this embodiment. In order to conveniently set track points on the trajectory path with reference to the discharge rate of the robot to avoid stacking, the discharge rate of the robot can be determined.

The inherent property of robot motion is also reflected in that if the robot sets the same motion speed parameters at the corner and the straight line, its motion speed at the corner and the straight line will be different, and the specific motion speed at the corner is slower than that at the straight line. In practice, another inherent property of the robot, the discharge rate is unchanged. Therefore, for the discharge rate and motion speed parameters of a suitable straight line, a stacking situation will occur at the corner. In some embodiments, on the premise of ensuring that the robot moves along the determined trajectory path, the distance between the trajectory points set at the corners on the trajectory path may be larger than the distance between the trajectory points set at the straight line, so as to achieve a straight line Balance the movement speed at the corner and the movement speed at the corner, so as to solve the stacking phenomenon that may be caused by the corner. A minimum distance can be set at the straight line to limit the distance between the trajectory points on the straight line to prevent the robot from jamming and stacking due to the excessive number of trajectory points on the straight line. It is also possible to set moving speed parameters with different values at the straight line and at the corner to achieve a balance between the moving speed at the straight line and the moving speed at the corner, and solve the problem of stacking due to inherent properties.

The walking sequence of the trajectory points is determined according to the initial pose of the robot, so as to obtain the trajectory point information. It can be understood that, in order to make the robot take fewer redundant trajectories, the initial point of the trajectory point is set as a point close to the initial pose of the robot, for example, it can be the trajectory point corresponding to the middle part of the object to be filled close to the side of the robot. That is to say, after the initial pose of the robot is determined, the track point corresponding to the middle point (or the track point closest to the point) on the trajectory path of the side closest to the initial pose of the robot can be used as The initial track point of the track point, after that, other track points can be moved clockwise, or other track points can be moved counterclockwise.

In some embodiments, the track point information may specifically include track point coordinates, initial track point coordinates, a position sequence of track points, and motion speed parameters of track points, and the like.

In some embodiments, the track point information further includes: normal information corresponding to the contour points.

Specifically, the normal information can be the angle value of the normal vector corresponding to each contour point cloud relative to a fixed amount, and can also be the deviation of the point cloud in the corresponding position sequence behind the contour point cloud relative to the previous point cloud. angle value.

Fig. 8 shows a schematic structural diagram of a groove filling device based on groove contour identification according to yet another embodiment of the present application, and the device includes:

The contour extraction module 500 is used for extracting the contour of the groove on the surface of the object, that is, for implementing step S100;

The contour identification module 600 is used to identify the type of the groove contour, that is, to realize step S200;

The filling scheme determination module 700 is used to determine the corresponding groove filling scheme according to the type of the groove profile, that is, to implement step S300;

The filling module 800 is used to fill the grooves based on the groove filling scheme, that is, to implement step S400.

Optionally, the contour extraction module 500 is also used to remove the non-groove contour, for example, it can be removed according to the number of pixel points.

Optionally, the types of groove contours that can be identified by the contour identification module 600 include at least one of closed grooves, open grooves, and single-point grooves.

Optionally, when the filling scheme determining module 700 determines to use a single-point groove filling scheme, the filling module 800 acquires the 3D point cloud of the single point, and fills the groove according to the pose of the 3D point cloud.

Optionally, when the filling scheme determination module 700 determines to use the closed groove and/or open groove filling scheme, the filling module 800 traverses the points on the inner contour or the outer contour according to the preset trajectory point interval, At a point, find the point from the point along the local normal direction to the corresponding outer or inner contour, and select the point between the point and the corresponding point on the outer or inner contour as the trajectory point. The trajectory point may be a midpoint between the traversed point and the corresponding point on the outer or inner contour.

Optionally, the filling module 800 determines the moving speed Vk of the filling tool at the track point according to the groove width Wk at the track point. A moving speed V0 corresponding to a specific width W0 may be preset, and the moving speed Vk at the track point is a linear or nonlinear ratio of V0 and Rk, where Rk=Wk/W0.

FIG. 9 shows a schematic structural diagram of a device for extracting a contour of a shallow groove on an object surface according to yet another embodiment of the present application, and the device includes:

The image acquisition module 510 is used for acquiring a two-dimensional image of the surface of the object, that is, for implementing step S110;

an image equalization module 520, configured to perform an image equalization operation on the image, that is, to implement step S120;

The image segmentation module 530 is configured to perform an image segmentation operation on the equalized image, that is, to implement step S130;

The contour extraction module 540 is configured to determine the shallow groove according to the segmented image and extract the contour of the shallow groove, that is, to implement step S140.

Optionally, the shallow groove contour extraction device further includes a three-dimensional image acquisition module for acquiring a three-dimensional image of the surface of the object.

Optionally, the image acquisition module 510 may acquire a two-dimensional image by mapping a three-dimensional image.

Optionally, the contour extraction module 540 is also used to remove the non-groove contour, for example, the non-groove contour can be removed according to the number of pixel points.

Optionally, the shallow groove contour includes an outermost groove contour and an inner groove contour, and the outermost groove contour includes an inner contour and an outer contour.

Optionally, the inner contour of the inner groove and the inner contour of the outermost groove may be extracted according to the two-dimensional image, and the inner contour of the outermost groove may be extracted according to the three-dimensional image.

10 shows a schematic structural diagram of a groove filling device based on robot movement speed control according to yet another embodiment of the present application, the device includes:

The contour extraction module 811 is used to extract the contour of the groove on the surface of the object, that is, to realize step S411;

The trajectory point determination module 812 is used to traverse the points on the contour according to the preset trajectory point interval, and determine the trajectory point of the robot moving at each traversed point, that is, to implement step S412;

The movement speed determination module 813 is configured to determine the robot movement speed Vk at the track point based on the groove width Wk at the track point, that is, to implement step S413;

The filling module 814 is used to make the robot perform groove filling according to the determined trajectory point and the movement speed at the trajectory point, that is, to implement step S414.

Optionally, the contour extraction module 811 is also used to remove the non-groove contour, for example, it can be removed according to the number of pixel points.

Optionally, the moving speed determination module 814 determines the moving speed Vk of the filling tool at the track point according to the groove width Wk at the track point. A moving speed V0 corresponding to a specific width W0 may be preset, and the moving speed Vk at the track point is a linear or nonlinear ratio of V0 and Rk, where Rk=Wk/W0.

Optionally, the trajectory point determination module 812 traverses the points on the inner contour or the outer contour according to the preset trajectory point interval, and at each point traversed, obtains the point along the local normal direction to the corresponding outer contour or the outer contour. point on the inner contour, and select the point between the point and the corresponding point on the outer or inner contour as the trajectory point. The trajectory point may be a midpoint between the traversed point and the corresponding point on the outer or inner contour.

11 shows a schematic structural diagram of a device for filling a non-closed groove on an object surface according to another embodiment of the present application, the device includes:

The contour extraction module 821 is used to extract the contour of the groove on the surface of the object, that is, to realize step S421;

The circumscribed rectangle generation module 822 generates the circumscribed rectangle of the outline, that is, is used to realize step S422;

The opening determination module 823 is used to determine the two opening end points of the outline based on the generated circumscribed rectangle, that is, to realize step S423;

The filling module 824 is used to perform groove filling using the two opening end points as the starting point and the ending point respectively, that is, to implement step S424.

Optionally, the fill module 824 breaks the groove profile at the start and end points to form separate inner and outer profiles.

Optionally, the filling module 824 determines the moving speed Vk of the filling tool at the track point according to the groove width Wk at the track point. A moving speed V0 corresponding to a specific width W0 may be preset, and the moving speed Vk at the track point is a linear or nonlinear ratio of V0 and Rk, where Rk=Wk/W0.

Optionally, the filling module 824 traverses the points on the inner contour or the outer contour according to the preset trajectory point interval, and at each point traversed, obtains the point along the local normal direction to the corresponding outer contour or inner contour. and select the point between the point and the corresponding point on the outer or inner contour as the trajectory point. The trajectory point may be a midpoint between the traversed point and the corresponding point on the outer or inner contour.

In the apparatus embodiments shown in FIG. 8 to FIG. 11 above, only the main functions of the modules are described, all the functions of each module correspond to the corresponding steps in the method embodiments, and the working principles of each module can also refer to the corresponding steps in the method embodiments. The description of the steps will not be repeated here. In addition, although the above-mentioned embodiment defines the corresponding relationship between the functions of the functional modules and the methods, those skilled in the art can understand that the functions of the functional modules are not limited to the above-mentioned corresponding relationships, that is, a specific functional module can also implement other method steps. or part of a method step. For example, the above embodiments describe the method for implementing step S200 by the contour identification module 600, however, the contour identification module 600 can also be used for implementing the method or a part of the method in steps S100, S300 or S400 according to actual needs.

The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the method of any one of the foregoing embodiments. It should be noted that the computer program stored in the computer-readable storage medium of the embodiments of the present application may be executed by the processor of the electronic device. In addition, the computer-readable storage medium may be a storage medium built in the electronic device, or a storage medium capable of The storage medium of the electronic device is pluggable and pluggable. Therefore, the computer-readable storage medium of the embodiments of the present application has high flexibility and reliability.

FIG. 12 shows a schematic structural diagram of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

As shown in FIG. 12 , the electronic device may include: a processor (processor) 902 , a communication interface (Communications Interface) 904 , a memory (memory) 906 , and a communication bus 908 .

in:

The processor 902 , the communication interface 904 , and the memory 906 communicate with each other through the communication bus 908 .

The communication interface 904 is used to communicate with network elements of other devices such as clients or other servers.

The processor 902 is configured to execute the program 910, and specifically may execute the relevant steps in the foregoing method embodiments.

Specifically, the program 910 may include program code including computer operation instructions.

The processor 902 may be a central processing unit (CPU), or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the electronic device may be the same type of processors, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.

The memory 906 is used to store the program 910 . Memory 906 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

The program 910 may specifically be used to cause the processor 902 to perform various operations in the foregoing method embodiments.

In general, the application content of this application includes:

A groove filling method based on groove contour recognition, comprising:

Extract the contour of the groove on the surface of the object;

Identify the type of groove profile;

Determine the corresponding groove filling scheme according to the type of groove profile;

The grooves are filled based on the groove filling scheme.

Optionally, the extracting the groove contour on the surface of the object includes removing the non-groove contour.

Optionally, the removing the non-groove contour includes removing the non-groove contour according to the number of pixel points.

Optionally, the groove types include closed grooves and/or open grooves.

Optionally, the groove type includes a single point groove.

Optionally, the groove profile includes an inner profile and an outer profile.

Optionally, traverse the points on the inner contour or the outer contour according to a preset trajectory point interval.

Optionally, at each traversed point, obtain the point from the point along the local normal direction to the corresponding outer contour or inner contour.

Optionally, at each traversed point, a point between the point and a point on the corresponding outer contour or inner contour is selected as a trajectory point.

Optionally, the trajectory point is a midpoint between the traversed point and a corresponding point on the outer contour or inner contour.

Optionally, the moving speed Vk of the filling tool at the track point is determined according to the groove width Wk at the track point.

Optionally, a moving speed V0 corresponding to a specific width W0 is preset, and the moving speed Vk at the trajectory point is a linear or nonlinear ratio of V0 and Rk, where Rk=Wk/W0.

Optionally, a 3D point cloud of the single point is obtained, and the groove is filled according to the pose of the 3D point cloud.

A method for extracting the contour of a shallow groove on the surface of an object, comprising:

Obtain a two-dimensional image of the surface of the object;

Perform image equalization operations on images;

Perform image segmentation on the equalized image;

Determine shallow grooves and extract shallow groove contours from the segmented image.

Optionally, it also includes acquiring a three-dimensional image of the surface of the object.

Optionally, the two-dimensional image is obtained by mapping a three-dimensional image.

Optionally, the determining the shallow groove and extracting the contour of the shallow groove includes removing the contour of the non-groove.

Optionally, the shallow groove contour includes an outermost groove contour and an inner groove contour.

Optionally, the inner layer groove contour is extracted according to a two-dimensional image.

Optionally, the outermost groove contour includes an inner contour and an outer contour.

Optionally, the inner contour of the outermost groove is extracted according to a two-dimensional image.

Optionally, the outer contour of the outermost groove is extracted according to a three-dimensional image.

A groove filling method based on robot movement speed control, comprising:

Extract the contour of the groove on the surface of the object;

Let the robot perform groove filling according to the determined trajectory point and the movement speed at the trajectory point.

Optionally, preset the movement speed V0 corresponding to the specific width W0, and calculate the robot movement speed Vk at the trajectory point according to W0, V0 and Wk.

Optionally, the moving speed Vk at the trajectory point is a linear or nonlinear ratio of V0 and Rk, where Rk=Wk/W0.

Optionally, the groove profile includes an inner profile and an outer profile.

Optionally, at each point traversed, obtain the point from the point along the local normal direction to the corresponding outer contour or inner contour, and select the point and the corresponding point on the outer contour or inner contour. The points in between are used as trajectory points.

A non-closed groove filling method on the surface of an object

Extract the contour of the non-closed groove on the surface of the object;

Generate the circumscribed rectangle of the outline;

Determine two open end points of the contour based on the generated circumscribed rectangle;

The groove filling is performed with the two open end points as the start and end points, respectively.

Optionally, the groove profile is broken at the start and end points to form separate inner and outer profiles.

Optionally, traverse the points on the inner contour or the outer contour from the start point to the end point according to the preset trajectory point interval.

In the description of this specification, reference to the terms "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" or the like is meant to be used in conjunction with the described embodiments. A particular feature, structure, material, or characteristic described in a manner or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

The logic and/or steps represented in flowcharts or otherwise described herein, for example, may be considered an ordered listing of executable instructions for implementing the logical functions, may be embodied in any computer-readable medium, For use by an instruction execution system, apparatus or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from and execute instructions from an instruction execution system, apparatus or apparatus), or in conjunction with such instruction execution system, apparatus or equipment. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport the program for use by or in connection with an instruction execution system, apparatus, or apparatus. More specific examples (non-exhaustive list) of computer readable media include the following: electrical connections with one or more wiring (electronic devices), portable computer disk cartridges (magnetic devices), random access memory (RAM), Read Only Memory (ROM), Erasable Editable Read Only Memory (EPROM or Flash Memory), Fiber Optic Devices, and Portable Compact Disc Read Only Memory (CDROM). In addition, the computer readable medium may even be paper or other suitable medium on which the program may be printed, as the paper or other medium may be optically scanned, for example, followed by editing, interpretation, or other suitable medium as necessary process to obtain the program electronically and then store it in computer memory.

The processor may be a Central Processing Unit (CPU), other general-purpose processors, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable processor Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It should be understood that various parts of the embodiments of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one or a combination of the following techniques known in the art: Discrete logic circuits, application specific integrated circuits with suitable combinational logic gates, Programmable Gate Arrays (PGA), Field Programmable Gate Arrays (FPGA), etc.

Those skilled in the art can understand that all or part of the steps carried by the methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program can be stored in a computer-readable storage medium. When executed, one or a combination of the steps of the method embodiment is included.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

The above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, and the like.

Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and alterations.

Claims

A groove filling method based on robot movement speed control, comprising:

Extract the contour of the groove on the surface of the object;

Traverse the points on the contour according to the preset trajectory point interval, and determine the trajectory point of the robot movement at each traversed point;

Determine the robot moving speed Vk at the track point based on the groove width Wk at the track point;

The groove filling is performed according to the determined track point and the movement speed at the track point.
The groove filling method according to claim 1, wherein: extracting the contour of the groove on the surface of the object comprises removing non-groove contours.
The groove filling method according to claim 2, wherein: removing the non-groove contour comprises removing the non-groove contour according to the number of pixel points.
The groove filling method according to claim 1, further comprising: presetting a moving speed V0 corresponding to a specific width W0, and calculating the robot moving speed Vk at the trajectory point according to W0, V0 and Wk.
The groove filling method according to claim 4, further comprising: the moving speed Vk at the track point is a linear or nonlinear ratio of V0 and Rk, wherein Rk=Wk/W0.
The groove filling method according to any one of claims 1-5, wherein: the groove contour includes an inner contour and an outer contour.
The groove filling method according to claim 6, wherein the determining the trajectory point of the robot movement according to the traversed points specifically comprises: at each traversed point, obtaining the point along the local normal direction to the corresponding The point on the outer or inner contour of the , select the point between the point and the corresponding point on the outer or inner contour as the trajectory point.
The groove filling method according to claim 7, wherein: the trajectory point is a midpoint between the traversed point and the corresponding point on the outer contour or the inner contour.
A groove filling device based on robot movement speed control, comprising:

The contour extraction module is used to extract the groove contour on the surface of the object;

The trajectory point determination module is used to traverse the points on the contour according to the preset trajectory point interval, and determine the trajectory point of the robot movement at each traversed point;

a movement speed determination module, configured to determine the robot movement speed Vk at the track point based on the groove width Wk at the track point;

The filling module is used to perform groove filling according to the determined trajectory point and the movement speed at the trajectory point.
An electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the computer program in claims 1 to 8 when the processor executes the computer program Any one of the groove filling methods based on robot movement speed control.
A computer-readable storage medium on which a computer program is stored, wherein when the computer program is executed by a processor, the groove filling method based on the control of the movement speed of a robot according to any one of claims 1 to 8 is implemented.